AI agents
Every endpoint is an MCP tool the agent calls by name.
Add Scrapier to Claude, Cursor or Windsurf with one command and the agent picks the right scraper on its own. Typed JSON it can act on directly. It also works from LangChain and n8n.
By the Scrapier team · Last updated
Six patterns cover most of what teams ship on Scrapier, from agents that call scrapers as tools to pipelines that refresh a dataset every night.
Six core jobs and the most common of the 22 others, with the data each collects and where it comes from.
| Use case | Data you collect | Example platforms | On this page |
|---|---|---|---|
| AI agent tool calls | Live profiles, posts and search results on request | Google, Reddit, YouTube, Instagram | ai-agents |
| RAG pipelines | Posts, threads, comments and transcripts for retrieval | Reddit, YouTube, any website | rag |
| Lead generation | Company and contact data from public listings | LinkedIn, Google Maps | lead-generation |
| Market research | Best sellers, reviews, jobs and category trends | Amazon, Glassdoor, TikTok | market-research |
| Brand monitoring | Mentions, comments and reviews | Reddit, Threads, YouTube, App Store | brand-monitoring |
| Price tracking | Prices, stock and deal discounts | Amazon, Walmart, eBay, Google Shopping | price-tracking |
| Ad intelligence | Active ads, creatives and run dates | Facebook Ad Library, Google Ads, TikTok | ad-intelligence |
| Video transcripts | The spoken text of videos | YouTube, TikTok, Instagram | video-transcripts |
| Review intelligence | Ratings and review text | App Store, Google Play, Amazon | review-intelligence |
| Local business discovery | Businesses, ratings and addresses | Google Maps | local-discovery |
The jobs most teams start with, the endpoints they call, and the shape of what comes back.
Every endpoint is an MCP tool the agent calls by name.
Add Scrapier to Claude, Cursor or Windsurf with one command and the agent picks the right scraper on its own. Typed JSON it can act on directly. It also works from LangChain and n8n.
Pull fresh posts, threads, reviews and docs into your knowledge base.
Consistent schemas across all 199 endpoints, so one ingestion job handles every source.
Build targeted lists from LinkedIn, Google Maps and company websites.
Hand-built lists go stale in weeks. Pull fresh company and contact rows on demand, as structured JSON ready to push into your CRM.
Monitor competitors, pricing and category trends through one API.
Surveys are slow and small. Best-seller lists, reviews, job postings and trending videos across Amazon, Walmart, Glassdoor and social platforms are fast and large.
Social listening: mentions, comments, reviews and sentiment inputs from one API key.
Mentions are scattered across Reddit, X, Threads, YouTube and app stores. One key, one shape.
Monitor product prices, stock and reviews across Amazon, Walmart, eBay and Google Shopping.
Prices move hourly, and every call scrapes live. See the method on our Amazon deal discounts data page.
22 more workloads built from the same endpoints.
22 results
Hand any public post URL to one endpoint and get its comments back, replies nested and pagination handled.
Resolve a handle, list its posts, then capture the engagement on each one.
Search creators by keyword or hashtag, confirm which market they post from, then score them before you spend anything.
Check one handle against every network in a single pass.
Surface the queries and hashtags climbing right now, then pull the posts driving them, without changing keys or schemas.
Pull the text of a video from the platform that hosts it, on the same schema for every network.
Follow an incident, a policy or a public institution across social channels and search from one query.
See what a competitor is paying to say: active ads, creatives and run dates.
Read what customers say about a product wherever they said it.
Match the same product across retailers by title, specs and images, so price and stock comparisons line up.
Mine reviews, forums and competitor listings for what a category asks for before you build.
Crawl a site into clean JSON so support bots answer from current content, not last year's.
Re-run the same endpoint on a schedule and diff the result to catch new listings, price edits and page changes.
Measure which hashtags actually drive reach: post volume, engagement and top posts per tag.
Find active local businesses by keyword and region, with ratings and addresses, then pull contact details from their websites.
Pull ranking pages for a keyword, qualify them, and extract contacts for outreach.
Compare follower growth, posting cadence and engagement against a set of competitor accounts.
Collect large, structured, deduplicated datasets for fine-tuning and evaluation.
Feed reviews, comments and posts into your sentiment model with source, date and rating attached.
Units sold, pricing, stock and ratings for any TikTok Shop product or seller.
Turn a destination into a list of short-term rentals with prices, ratings, reviews and occupancy.
Find every new video and comment that mentions your brand or product, without manual searching.
Start here, then browse the full catalogue for the long tail.
| Use case | Typical platforms | Start with | Typical output | How teams call it |
|---|---|---|---|---|
| AI agents | Any of 35 | /google-search/ | Typed JSON per tool call: title, url, snippet, position | MCP |
| RAG | Reddit, YouTube | /reddit-post-search/ | Posts, comments, transcripts | Scheduled REST jobs |
| Lead generation | LinkedIn, Google Maps | /google-maps/ | Businesses, profiles, contacts: name, rating, address | REST, then your CRM |
| Market research | Amazon, Glassdoor, TikTok | /amazon-bestsellers/ | Products, reviews, ratings | REST or MCP |
| Brand monitoring | Reddit, Threads, YouTube | /threads-search/ | Mentions, replies, reviews | Scheduled REST jobs |
| Price tracking | Amazon, Walmart, eBay | /amazon-todays-deals/ | Price, basis price, savings, stock | Scheduled REST jobs |
| Industry | Common job | Data sources |
|---|---|---|
| E-commerce and retail | Price and stock monitoring | Amazon, Walmart, eBay, Target, Etsy |
| B2B sales and recruiting | Lead lists, job postings | LinkedIn, Google Maps, Glassdoor, Upwork, Seek |
| Marketing and agencies | Social listening, creator research | Instagram, TikTok, YouTube, Reddit |
| SEO | Rank and SERP tracking | Google, Bing, DuckDuckGo |
| Real estate and travel | Listing and price data | Zillow, Realtor.com, Idealista, Airbnb |
| AI products | Agent tools, RAG ingestion | Any endpoint via MCP |
Tracking 500 Amazon products daily for 30 days = 15,000 calls to /amazon-product/ at 2 credits = 30,000 credits. That fits the Pro plan at $79 with 50,000 credits.
Agents that read the live web, RAG pipelines, lead lists, competitor trackers, brand monitors and price trackers: anything that needs structured public data from 35 platforms.
No. One key covers all 199 endpoints across every platform, over REST and MCP.
Start with a use case if you know the job but not the sources. Start with a platform page if you already know where the data lives.
Yes. Every endpoint shares the same key, billing and response conventions, so mixing them adds no extra integration work.
Each endpoint has a fixed credit price per successful call, listed in the catalogue, however many results the call returns. A Google Maps search that returns a page of places is one call at 1 credit. Failed calls cost 0. See pricing.
A retailer tracking Amazon prices each morning; a sales team listing companies from LinkedIn; an agent pulling Reddit threads to answer a question; a marketer collecting TikTok comments about a brand.
Yes. Through MCP, an agent in Claude or Cursor calls a scraping endpoint as a tool and receives structured JSON it reads directly, with no browser or parsing code.
Every plan includes all 199 endpoints. You pay per successful call.