New: the Scrapier MCP server. Every scraper in Claude Code with one command. See how →
Use cases

What People Build With Scrapier

By the Scrapier team · Last updated

Six patterns cover most of what teams ship on Scrapier, from agents that call scrapers as tools to pipelines that refresh a dataset every night.

Quick answerWhat is web scraping used for? Giving AI agents live web data, feeding RAG indexes, building lead lists, tracking competitors, monitoring brand mentions and tracking prices. Scrapier covers all six with 199 endpoints across 35 platforms that return typed JSON over REST or MCP.
At a glance

Web scraping use cases at a glance

Six core jobs and the most common of the 22 others, with the data each collects and where it comes from.

Use caseData you collectExample platformsOn this page
AI agent tool callsLive profiles, posts and search results on requestGoogle, Reddit, YouTube, Instagramai-agents
RAG pipelinesPosts, threads, comments and transcripts for retrievalReddit, YouTube, any websiterag
Lead generationCompany and contact data from public listingsLinkedIn, Google Mapslead-generation
Market researchBest sellers, reviews, jobs and category trendsAmazon, Glassdoor, TikTokmarket-research
Brand monitoringMentions, comments and reviewsReddit, Threads, YouTube, App Storebrand-monitoring
Price trackingPrices, stock and deal discountsAmazon, Walmart, eBay, Google Shoppingprice-tracking
Ad intelligenceActive ads, creatives and run datesFacebook Ad Library, Google Ads, TikTokad-intelligence
Video transcriptsThe spoken text of videosYouTube, TikTok, Instagramvideo-transcripts
Review intelligenceRatings and review textApp Store, Google Play, Amazonreview-intelligence
Local business discoveryBusinesses, ratings and addressesGoogle Mapslocal-discovery
Core

Six core use cases

The jobs most teams start with, the endpoints they call, and the shape of what comes back.

01

AI agents

Every endpoint is an MCP tool the agent calls by name.

Add Scrapier to Claude, Cursor or Windsurf with one command and the agent picks the right scraper on its own. Typed JSON it can act on directly. It also works from LangChain and n8n.

ClaudeCursorWindsurfLangChainn8n
Read the AI agents guide →
Example
$ claude mcp add scrapier ✓ scrapier connected > Find 5 coffee roasters in Austin with 4.5+ stars → google_maps_search(query, min_rating=4.5) { "results": 5 }
02

RAG pipelines

Pull fresh posts, threads, reviews and docs into your knowledge base.

Consistent schemas across all 199 endpoints, so one ingestion job handles every source.

/reddit-post-search//reddit-comments//youtube-subtitles//website-content-crawler/
Browse the API reference →
Example
GET /youtube-subtitles/?urls=…watch?v=dQw4w9WgXcQ { "title": "Rick Astley - Never Gonna Give You Up…", "language_code": "en", "duration_seconds": 213 }
03

Lead generation

Build targeted lists from LinkedIn, Google Maps and company websites.

Hand-built lists go stale in weeks. Pull fresh company and contact rows on demand, as structured JSON ready to push into your CRM.

/linkedin-company-about//linkedin-profile//google-maps//contact-extractor/
Browse the API reference →
Example
GET /linkedin-company-about/?urls=…/company/microsoft { "name": "Microsoft", "industry": "Software Development", "employees": "10,001+", "headquarters": "Redmond, WA" }
04

Market research

Monitor competitors, pricing and category trends through one API.

Surveys are slow and small. Best-seller lists, reviews, job postings and trending videos across Amazon, Walmart, Glassdoor and social platforms are fast and large.

/amazon-bestsellers//amazon-review//glassdoor-jobs//tiktok-trending-videos/
See our Amazon best-seller data →
Example
GET /tiktok-trending-videos/?country_code=US { "title": "3 NEW Gemini Live Features…", "playCount": 66100000, "diggCount": 23700 }
05

Brand monitoring

Social listening: mentions, comments, reviews and sentiment inputs from one API key.

Mentions are scattered across Reddit, X, Threads, YouTube and app stores. One key, one shape.

/reddit-post-search//threads-search//youtube-comments-scraper//apple-app-store-review-scraper/
Browse the API reference →
Example
GET /threads-search/?query=AI { "username": "theeconomist", "likeCount": 1240, "directReplyCount": 87, "takenAtISO": "2026-06-17T12:40:28Z" }
06

Price tracking

Monitor product prices, stock and reviews across Amazon, Walmart, eBay and Google Shopping.

Prices move hourly, and every call scrapes live. See the method on our Amazon deal discounts data page.

/amazon-product//amazon-todays-deals//walmart-data-extractor//ebay-product-details-scraper/
Browse the API reference →
Example
Retailer Price Stock Amazon $49.99 In stock Walmart $52.49 In stock eBay $47.20 3 left
More workloads

What else do teams run on Scrapier?

22 more workloads built from the same endpoints.

22 results

Comments

Comment scraping

Hand any public post URL to one endpoint and get its comments back, replies nested and pagination handled.

InstagramTikTokYouTubeReddit
Creators

Creator analytics

Resolve a handle, list its posts, then capture the engagement on each one.

InstagramTikTokYouTubeX
Influencers

Influencer discovery

Search creators by keyword or hashtag, confirm which market they post from, then score them before you spend anything.

TikTokInstagramYouTube
Identity

Identity resolution

Check one handle against every network in a single pass.

InstagramTikTokXThreads
Trends

Trend detection

Surface the queries and hashtags climbing right now, then pull the posts driving them, without changing keys or schemas.

GoogleTikTokRedditYouTube
Transcripts

Video transcripts

Pull the text of a video from the platform that hosts it, on the same schema for every network.

YouTubeTikTokInstagram
OSINT

News and OSINT monitoring

Follow an incident, a policy or a public institution across social channels and search from one query.

GoogleXRedditTelegram
Ads

Ad intelligence

See what a competitor is paying to say: active ads, creatives and run dates.

Facebook Ad LibraryGoogle AdsTikTokLinkedIn
Reviews

Review intelligence

Read what customers say about a product wherever they said it.

AmazonApp StoreGoogle PlayEtsy
Matching

Product matching

Match the same product across retailers by title, specs and images, so price and stock comparisons line up.

AmazonWalmarteBayTarget
Product

Product development

Mine reviews, forums and competitor listings for what a category asks for before you build.

RedditAmazonApp StoreEtsy
Chatbots

Turn any website into chatbot-ready data

Crawl a site into clean JSON so support bots answer from current content, not last year's.

Any websiteDocs sitesHelp centers
Changes

Web data monitoring

Re-run the same endpoint on a schedule and diff the result to catch new listings, price edits and page changes.

Any websiteAmazonLinkedIn
Hashtags

Hashtag research

Measure which hashtags actually drive reach: post volume, engagement and top posts per tag.

InstagramTikTokYouTube
Local

Local business discovery

Find active local businesses by keyword and region, with ratings and addresses, then pull contact details from their websites.

Google MapsAny website
Benchmark

Competitor benchmarking

Compare follower growth, posting cadence and engagement against a set of competitor accounts.

InstagramFacebookLinkedInTikTok
Training

Training data for ML

Collect large, structured, deduplicated datasets for fine-tuning and evaluation.

RedditYouTubeAny website
Sentiment

Sentiment analysis

Feed reviews, comments and posts into your sentiment model with source, date and rating attached.

RedditXAmazonGoogle Play
TikTok Shop

TikTok Shop intelligence

Units sold, pricing, stock and ratings for any TikTok Shop product or seller.

TikTok Shop
Travel

Accommodation host discovery

Turn a destination into a list of short-term rentals with prices, ratings, reviews and occupancy.

Airbnb
YouTube

YouTube brand mentions

Find every new video and comment that mentions your brand or product, without manual searching.

YouTube
Endpoints

Which endpoints does each use case need?

Start here, then browse the full catalogue for the long tail.

Use caseTypical platformsStart withTypical outputHow teams call it
AI agentsAny of 35/google-search/Typed JSON per tool call: title, url, snippet, positionMCP
RAGReddit, YouTube/reddit-post-search/Posts, comments, transcriptsScheduled REST jobs
Lead generationLinkedIn, Google Maps/google-maps/Businesses, profiles, contacts: name, rating, addressREST, then your CRM
Market researchAmazon, Glassdoor, TikTok/amazon-bestsellers/Products, reviews, ratingsREST or MCP
Brand monitoringReddit, Threads, YouTube/threads-search/Mentions, replies, reviewsScheduled REST jobs
Price trackingAmazon, Walmart, eBay/amazon-todays-deals/Price, basis price, savings, stockScheduled REST jobs
Industries

Use cases by industry

IndustryCommon jobData sources
E-commerce and retailPrice and stock monitoringAmazon, Walmart, eBay, Target, Etsy
B2B sales and recruitingLead lists, job postingsLinkedIn, Google Maps, Glassdoor, Upwork, Seek
Marketing and agenciesSocial listening, creator researchInstagram, TikTok, YouTube, Reddit
SEORank and SERP trackingGoogle, Bing, DuckDuckGo
Real estate and travelListing and price dataZillow, Realtor.com, Idealista, Airbnb
AI productsAgent tools, RAG ingestionAny endpoint via MCP
Cost

Cost of a typical job

Tracking 500 Amazon products daily for 30 days = 15,000 calls to /amazon-product/ at 2 credits = 30,000 credits. That fits the Pro plan at $79 with 50,000 credits.

Failed calls cost 0 credits and are excluded.
See pricing
Limits

Data and legal limits

  • Public data only; no private accounts or messages
  • Some endpoints take your own session cookie; the docs say which
  • Platform terms and privacy laws such as GDPR or CCPA apply when you store personal data, such as names in lead lists
  • Catalogue platforms only; the free web tools handle single pages
FAQ

Questions

What can I actually build with Scrapier?

Agents that read the live web, RAG pipelines, lead lists, competitor trackers, brand monitors and price trackers: anything that needs structured public data from 35 platforms.

Do I need a different API key per platform?

No. One key covers all 199 endpoints across every platform, over REST and MCP.

Should I start from a use case or a platform page?

Start with a use case if you know the job but not the sources. Start with a platform page if you already know where the data lives.

Can I combine several use cases in one product?

Yes. Every endpoint shares the same key, billing and response conventions, so mixing them adds no extra integration work.

What does a use case cost to run?

Each endpoint has a fixed credit price per successful call, listed in the catalogue, however many results the call returns. A Google Maps search that returns a page of places is one call at 1 credit. Failed calls cost 0. See pricing.

What are examples of web scraping?

A retailer tracking Amazon prices each morning; a sales team listing companies from LinkedIn; an agent pulling Reddit threads to answer a question; a marketer collecting TikTok comments about a brand.

Can AI agents use web scraping?

Yes. Through MCP, an agent in Claude or Cursor calls a scraping endpoint as a tool and receives structured JSON it reads directly, with no browser or parsing code.

Get started

Pick a use case. Make the first call today.

Every plan includes all 199 endpoints. You pay per successful call.