# What People Build With Scrapier — 2026 (https://scrapier.io/use-cases/)

> Scrapier powers AI agent tool use, RAG ingestion, lead generation, market research, brand monitoring and price tracking.

Last updated: 2026-11-10

Use cases

## What People Build _With Scrapier_

By the Scrapier team · Last updated 24 Oct 2026

Six patterns cover most of what teams ship on Scrapier, from agents that call scrapers as tools to pipelines that refresh a dataset every night.

Quick answer**What is web scraping used for?** Giving AI agents live web data, feeding RAG indexes, building lead lists, tracking competitors, monitoring brand mentions and tracking prices. Scrapier covers all six with 199 endpoints across 35 platforms that return typed JSON over REST or MCP.

At a glance

## Web scraping use cases at a glance

Six core jobs and the most common of the 22 others, with the data each collects and where it comes from.

| Use case | Data you collect | Example platforms | On this page |
| --- | --- | --- | --- |
| AI agent tool calls | Live profiles, posts and search results on request | Google, Reddit, YouTube, Instagram | [ai-agents](#ai-agents) |
| RAG pipelines | Posts, threads, comments and transcripts for retrieval | Reddit, YouTube, any website | [rag](#rag) |
| Lead generation | Company and contact data from public listings | LinkedIn, Google Maps | [lead-generation](#lead-generation) |
| Market research | Best sellers, reviews, jobs and category trends | Amazon, Glassdoor, TikTok | [market-research](#market-research) |
| Brand monitoring | Mentions, comments and reviews | Reddit, Threads, YouTube, App Store | [brand-monitoring](#brand-monitoring) |
| Price tracking | Prices, stock and deal discounts | Amazon, Walmart, eBay, Google Shopping | [price-tracking](#price-tracking) |
| Ad intelligence | Active ads, creatives and run dates | Facebook Ad Library, Google Ads, TikTok | [ad-intelligence](#ad-intelligence) |
| Video transcripts | The spoken text of videos | YouTube, TikTok, Instagram | [video-transcripts](#video-transcripts) |
| Review intelligence | Ratings and review text | App Store, Google Play, Amazon | [review-intelligence](#review-intelligence) |
| Local business discovery | Businesses, ratings and addresses | Google Maps | [local-discovery](#local-discovery) |

Core

## Six core use cases

The jobs most teams start with, the endpoints they call, and the shape of what comes back.

01

### AI agents

Every endpoint is an MCP tool the agent calls by name.

Add Scrapier to Claude, Cursor or Windsurf with one command and the agent picks the right scraper on its own. Typed JSON it can act on directly. It also works from LangChain and n8n.

ClaudeCursorWindsurfLangChainn8n

[Read the AI agents guide →](https://docs.scrapier.io/ai/mcp/)

Example

$ claude mcp add scrapier ✓ scrapier connected > Find 5 coffee roasters in Austin with 4.5+ stars → google\_maps\_search(query, min\_rating=4.5) { "results": 5 }

02

### RAG pipelines

Pull fresh posts, threads, reviews and docs into your knowledge base.

Consistent schemas across all 199 endpoints, so one ingestion job handles every source.

/reddit-post-search//reddit-comments//youtube-subtitles//website-content-crawler/

[Browse the API reference →](https://docs.scrapier.io/api-reference/)

Example

GET /youtube-subtitles/?urls=…watch?v=dQw4w9WgXcQ { "title": "Rick Astley - Never Gonna Give You Up…", "language\_code": "en", "duration\_seconds": 213 }

03

### Lead generation

Build targeted lists from LinkedIn, Google Maps and company websites.

Hand-built lists go stale in weeks. Pull fresh company and contact rows on demand, as structured JSON ready to push into your CRM.

/linkedin-company-about//linkedin-profile//google-maps//contact-extractor/

[Browse the API reference →](https://docs.scrapier.io/api-reference/)

Example

GET /linkedin-company-about/?urls=…/company/microsoft { "name": "Microsoft", "industry": "Software Development", "employees": "10,001+", "headquarters": "Redmond, WA" }

04

### Market research

Monitor competitors, pricing and category trends through one API.

Surveys are slow and small. Best-seller lists, reviews, job postings and trending videos across Amazon, Walmart, Glassdoor and social platforms are fast and large.

/amazon-bestsellers//amazon-review//glassdoor-jobs//tiktok-trending-videos/

[See our Amazon best-seller data →](https://scrapier.io/data/amazon-best-seller-reviews/)

Example

GET /tiktok-trending-videos/?country\_code=US { "title": "3 NEW Gemini Live Features…", "playCount": 66100000, "diggCount": 23700 }

05

### Brand monitoring

Social listening: mentions, comments, reviews and sentiment inputs from one API key.

Mentions are scattered across Reddit, X, Threads, YouTube and app stores. One key, one shape.

/reddit-post-search//threads-search//youtube-comments-scraper//apple-app-store-review-scraper/

[Browse the API reference →](https://docs.scrapier.io/api-reference/)

Example

GET /threads-search/?query=AI { "username": "theeconomist", "likeCount": 1240, "directReplyCount": 87, "takenAtISO": "2026-06-17T12:40:28Z" }

06

### Price tracking

Monitor product prices, stock and reviews across Amazon, Walmart, eBay and Google Shopping.

Prices move hourly, and every call scrapes live. See the method on our [Amazon deal discounts](https://scrapier.io/data/amazon-deal-discounts/) data page.

/amazon-product//amazon-todays-deals//walmart-data-extractor//ebay-product-details-scraper/

[Browse the API reference →](https://docs.scrapier.io/api-reference/)

Example

Retailer Price Stock Amazon $49.99 In stock Walmart $52.49 In stock eBay $47.20 3 left

More workloads

## What else do teams run on Scrapier?

22 more workloads built from the same endpoints.

Search use cases

22 results

Comments

### Comment scraping

Hand any public post URL to one endpoint and get its comments back, replies nested and pagination handled.

InstagramTikTokYouTubeReddit

Creators

### Creator analytics

Resolve a handle, list its posts, then capture the engagement on each one.

InstagramTikTokYouTubeX

Influencers

### Influencer discovery

Search creators by keyword or hashtag, confirm which market they post from, then score them before you spend anything.

TikTokInstagramYouTube

Identity

### Identity resolution

Check one handle against every network in a single pass.

InstagramTikTokXThreads

Trends

### Trend detection

Surface the queries and hashtags climbing right now, then pull the posts driving them, without changing keys or schemas.

GoogleTikTokRedditYouTube

Transcripts

### Video transcripts

Pull the text of a video from the platform that hosts it, on the same schema for every network.

YouTubeTikTokInstagram

OSINT

### News and OSINT monitoring

Follow an incident, a policy or a public institution across social channels and search from one query.

GoogleXRedditTelegram

Ads

### Ad intelligence

See what a competitor is paying to say: active ads, creatives and run dates.

Facebook Ad LibraryGoogle AdsTikTokLinkedIn

Reviews

### Review intelligence

Read what customers say about a product wherever they said it.

AmazonApp StoreGoogle PlayEtsy

Matching

### Product matching

Match the same product across retailers by title, specs and images, so price and stock comparisons line up.

AmazonWalmarteBayTarget

Product

### Product development

Mine reviews, forums and competitor listings for what a category asks for before you build.

RedditAmazonApp StoreEtsy

Chatbots

### Turn any website into chatbot-ready data

Crawl a site into clean JSON so support bots answer from current content, not last year's.

Any websiteDocs sitesHelp centers

Changes

### Web data monitoring

Re-run the same endpoint on a schedule and diff the result to catch new listings, price edits and page changes.

Any websiteAmazonLinkedIn

Hashtags

### Hashtag research

Measure which hashtags actually drive reach: post volume, engagement and top posts per tag.

InstagramTikTokYouTube

Local

### Local business discovery

Find active local businesses by keyword and region, with ratings and addresses, then pull contact details from their websites.

Google MapsAny website

SEO

### Link prospecting

Pull ranking pages for a keyword, qualify them, and extract contacts for outreach.

GoogleBingAny website

Benchmark

### Competitor benchmarking

Compare follower growth, posting cadence and engagement against a set of competitor accounts.

InstagramFacebookLinkedInTikTok

Training

### Training data for ML

Collect large, structured, deduplicated datasets for fine-tuning and evaluation.

RedditYouTubeAny website

Sentiment

### Sentiment analysis

Feed reviews, comments and posts into your sentiment model with source, date and rating attached.

RedditXAmazonGoogle Play

TikTok Shop

### TikTok Shop intelligence

Units sold, pricing, stock and ratings for any TikTok Shop product or seller.

TikTok Shop

Travel

### Accommodation host discovery

Turn a destination into a list of short-term rentals with prices, ratings, reviews and occupancy.

Airbnb

YouTube

### YouTube brand mentions

Find every new video and comment that mentions your brand or product, without manual searching.

YouTube

Endpoints

## Which endpoints does each use case need?

Start here, then [browse the full catalogue](https://scrapier.io/scrapers/) for the long tail.

| Use case | Typical platforms | Start with | Typical output | How teams call it |
| --- | --- | --- | --- | --- |
| AI agents | Any of 35 | `/google-search/` | Typed JSON per tool call: title, url, snippet, position | MCP |
| RAG | Reddit, YouTube | `/reddit-post-search/` | Posts, comments, transcripts | Scheduled REST jobs |
| Lead generation | LinkedIn, Google Maps | `/google-maps/` | Businesses, profiles, contacts: name, rating, address | REST, then your CRM |
| Market research | Amazon, Glassdoor, TikTok | `/amazon-bestsellers/` | Products, reviews, ratings | REST or MCP |
| Brand monitoring | Reddit, Threads, YouTube | `/threads-search/` | Mentions, replies, reviews | Scheduled REST jobs |
| Price tracking | Amazon, Walmart, eBay | `/amazon-todays-deals/` | Price, basis price, savings, stock | Scheduled REST jobs |

Industries

## Use cases by industry

| Industry | Common job | Data sources |
| --- | --- | --- |
| E-commerce and retail | Price and stock monitoring | Amazon, Walmart, eBay, Target, Etsy |
| B2B sales and recruiting | Lead lists, job postings | LinkedIn, Google Maps, Glassdoor, Upwork, Seek |
| Marketing and agencies | Social listening, creator research | Instagram, TikTok, YouTube, Reddit |
| SEO | Rank and SERP tracking | Google, Bing, DuckDuckGo |
| Real estate and travel | Listing and price data | Zillow, Realtor.com, Idealista, Airbnb |
| AI products | Agent tools, RAG ingestion | Any endpoint via MCP |

Cost

## Cost of a typical job

Tracking _500 Amazon products daily_ for 30 days = 15,000 calls to `/amazon-product/` at 2 credits = 30,000 credits. That fits the _Pro plan at $79_ with 50,000 credits.

Failed calls cost 0 credits and are excluded.

[See pricing](https://scrapier.io/pricing/)

Limits

## Data and legal limits

- Public data only; no private accounts or messages
- Some endpoints take your own session cookie; the docs say which
- Platform terms and privacy laws such as GDPR or CCPA apply when you store personal data, such as names in lead lists
- Catalogue platforms only; the free [web tools](https://scrapier.io/tools/) handle single pages

FAQ

## Questions

### What can I actually build with Scrapier?

Agents that read the live web, RAG pipelines, lead lists, competitor trackers, brand monitors and price trackers: anything that needs structured public data from 35 platforms.

### Do I need a different API key per platform?

No. One key covers all 199 endpoints across every platform, over REST and MCP.

### Should I start from a use case or a platform page?

Start with a use case if you know the job but not the sources. Start with a platform page if you already know where the data lives.

### Can I combine several use cases in one product?

Yes. Every endpoint shares the same key, billing and response conventions, so mixing them adds no extra integration work.

### What does a use case cost to run?

Each endpoint has a fixed credit price per successful call, listed in the catalogue, however many results the call returns. A Google Maps search that returns a page of places is one call at 1 credit. Failed calls cost 0. See [pricing](https://scrapier.io/pricing/).

### What are examples of web scraping?

A retailer tracking Amazon prices each morning; a sales team listing companies from LinkedIn; an agent pulling Reddit threads to answer a question; a marketer collecting TikTok comments about a brand.

### Can AI agents use web scraping?

Yes. Through MCP, an agent in Claude or Cursor calls a scraping endpoint as a tool and receives structured JSON it reads directly, with no browser or parsing code.

Get started

## Pick a use case. Make the first call today.

Every plan includes all 199 endpoints. You pay per successful call.

[Get your API key](https://app.scrapier.io/sign-up) [See pricing](https://scrapier.io/pricing/)

---
Canonical page: https://scrapier.io/use-cases/
Site index for AI agents: https://scrapier.io/llms.txt
