Yahoo Search Scraper
The results Yahoo shows for a query as typed JSON: URL, title, description, page text, logo, sub-links and domain, filtered by recency if you want. 1 credit per page. No Yahoo login, no proxies to manage.
Free API key · 1 credit per page of results · failed calls free
// GET /api/v2/yahoo-search/?queries=java developer · sample from the catalog { "success": true, "data": { "query": "java developer", "page": 1, "page_size": 7, "results": [ { "url": "https://www.geeksforgeeks.org/gfg-academy/how-to-become-a-java-developer/", "title": "How to become a Java Developer? - GeeksforGeeks", "description": "What is a Java Developer? A Java Developer is a software engineer who builds, maintains, and improves applications using Java.", "text": " * Careers & Jobs in Java\n\nCareers in Java are in high demand...\n", "logo_url": "https://s.yimg.com/pv/static/search_favicon/images/32x32_7eae5aac8b7f7402.png", "links": [ "https://www.geeksforgeeks.org/java/careers-jobs-in-java/" ], "domain": "www.geeksforgeeks.org" } ], "next_cursor": "eyJxIjoiamF2YSBkZXZlbG9wZXIiLCJiIjo4LCJ0cCI6IkFueXRpbWUiLCJiZiI6dHJ1ZSwiYmMiOjgsImJsIjoxMCwiaCI6ImFiYzEyM2RlZiJ9" } }
| field | value | type | Has a value✓ |
|---|
Scrapier's Yahoo Search Scraper returns the results Yahoo shows for a query as JSON, a page at a time: url, title, description, text, logo_url, links and domain. It costs 1 credit per successful call. No Yahoo login is needed.
Yahoo Search Scraper extracts the results page Yahoo returns for a search term, one page per call: each result's destination URL, title, description, extracted page text, favicon, sub-links and domain, plus the page number and the query it answered. Filter by recency with <code>time_period</code> and page through with a cursor. Every response is structured JSON, ready to pass directly to an LLM, index into a vector store or feed a research pipeline. It is a Yahoo data scraper on the same API key used by 12,000+ developers. Next, follow next_cursor for page two.
What data can I extract with Yahoo Search Scraper?
Using this Yahoo Search Scraper, you can extract the following data from public Yahoo pages of results:
Want the exact field names? See the example output.
Try Yahoo Search Scraper Live
Press Run to see the catalog sample. Add your API key and it runs a real call to the Scrapier API from your browser. The key is sent only to proxy.scrapier.io and never stored.
+ Run live with your API key
How to use Yahoo Search Scraper
Hand it to your AI assistant, or run it yourself in five steps.
Copy to your AI assistant
Paste this into ChatGPT, Claude or Cursor and it knows how to call the scraper.
Use the Scrapier Yahoo Search Scraper to get public Yahoo pages of results. Endpoint: GET https://proxy.scrapier.io/api/v2/yahoo-search/?queries={queries} with header "X-API-Key: YOUR_API_KEY" and a 210-second timeout. Send one value per call in queries; for a list, make one call per value, at most 3 in parallel. It returns JSON under data.results: url, title, description, text, logo_url, links, domain. Each successful call costs 1 credit; failed calls are free. If you support MCP, run: claude mcp add -t http scrapier https://mcp.scrapier.io/mcp --header "X-API-Key: YOUR_API_KEY" and call run_scrape with endpoint "yahoo-search" (find it with search_endpoints).Use this scraper from ChatGPT, Claude or Cursor. Point your assistant at Scrapier and it reads Yahoo pages of results on its own: no glue code, no scraping logic in your prompts.
Run it yourself
- Get your free Scrapier API keySign up at app.scrapier.io; the free tier lets you try it first.
- Open Yahoo Search Scraper in the catalogIt lives under Scrapers / Yahoo, next to the other Yahoo tools.
- Enter the search termFor example
java developerinqueries. Settime_periodto Anytime, Past day, Past week or Past month to filter by recency. - Send your API requestOne GET with your key in the
X-API-Keyheader, or onerun_scrapecall from your agent. This is how to scrape Yahoo without a browser. Set a 210-second client timeout. - Save your resultsEach page comes back as JSON in
data.results: store it as is, or flatten it to CSV. Passdata.next_cursorback ascursorto get the next page until it is null.
How to scale to bulk Yahoo pages of results
queries takes one term per call, so for a keyword list you loop: one request per term. Within a term, pass back data.next_cursor unchanged to read the next page. Run up to 3 requests in parallel per API key; above that the API answers 429 with a Retry-After header, and throttled calls cost nothing. Each page is 1 credit and failed calls are free, so 1,000 pages cost 1,000 credits, inside the Hobby plan's 15,000.
Features
This Yahoo web scraper turns a Yahoo results page into rows you can rank, filter and store. It suits research, news and trend monitoring, lead discovery, content gap analysis and any workflow that needs Yahoo results without parsing HTML.
- Results
url,titleanddescriptionfor every result - Page text
textcarries the extracted text of the result page - Recency
time_periodlimits results to the past day, week or month - Sub-links
linkslists links found under each result - Paging
next_cursorreads the next page of results - Raw mode
raw_datareturns Yahoo's upstream payload instead
Proxies, retries and parsing run on Scrapier's side, so you send one request and get JSON back, at 1 credit per page of results and 0 credits for a failed call.
Use cases
- Market researcher searches a product category with
time_periodset to Past month and collects the titles, domains and descriptions of what is being published. - Content strategist runs a list of topic queries and reads
textandtitleto see how existing pages cover each topic. - Sales ops searches for a niche and collects the result URLs and domains as a lead list for enrichment.
- Competitive intel tracks which competitor pages show up for category queries and how their descriptions change over time.
- AI engineer gives an agent a tool that searches Yahoo, so it can ground answers in current results and cite the URLs.
Running these from an agent? Use the MCP integration and let Claude or Cursor call the scraper directly.
Why choose this Scraper API?
Each result can carry the text and sub-links of its own page, so one call gives you material to read, not only a list of URLs.
Each result arrives as typed fields with url, title, description and domain, so you skip selectors and layout changes.
The same key and response envelope work across Scrapier's endpoints, so search results can feed the next call without new code.
Input
The scraper takes one search term per call in queries. It is required on the first call and ignored when cursor is supplied. time_period is optional and locked in the cursor. Send parameters in the query string (GET) or as a JSON body (POST).
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
queries | string | Yes | — | A search term. One per call. Required on the first call; ignored when cursor is supplied. Aliases: query, q |
cursor | string | No | — | Pass back data.next_cursor from the previous response, unchanged, to get the next page; null means the last page |
time_period | string | No | Anytime | Filter by recency, default Anytime. Options: Anytime, Past day, Past week, Past month. Locked in the cursor |
backfill_empty_results | boolean | No | true | Fetch each result's own page to fill in text and links when Yahoo's snippet is empty, default true. Locked in the cursor |
backfill_concurrency | integer | No | 8 | Concurrent backfill page fetches per page, default 8. Locked in the cursor |
backfill_max_links | integer | No | 10 | Maximum sub-links to extract per backfilled result page, default 10. Locked in the cursor |
use_proxy | boolean | No | true | Route through Scrapier's residential proxy pool, on by default. Leave as is |
raw_data | boolean | No | false | Return Yahoo's raw upstream payload under raw_responses instead of the formatted items |
One term per call: to search ten terms, make ten calls and page each with cursor. Booleans are the strings true or false.
Example input
{
"queries": "java developer",
"time_period": "Anytime",
"backfill_empty_results": "true",
"backfill_concurrency": "8",
"backfill_max_links": "10",
"use_proxy": "true",
"raw_data": "false"
}Error codes
| Status | Meaning | Billed | Fix |
|---|---|---|---|
| 200 | Page of results data returned in data.results | Yes | — |
| 400 | Bad or missing input | No | Check queries: one value per call |
| 401 | X-API-Key header missing | No | Add the header |
| 402 | Out of credits (body has balance) | No | Top up; retrying will not help |
| 403 | Invalid or revoked API key | No | Check the key in your dashboard |
| 429 | Throttled | No | Wait Retry-After seconds, then retry |
| 5xx | Upstream failure or timeout | No | Retry with backoff; set a 210 s client timeout |
From the error reference. Any non-2xx response costs 0 credits.
Output
Typed JSON with the same schema on every call. One call returns a page of results under results, each with 7 fields, inside the standard data envelope. Field names stay the same on every call. Set raw_data to true if you want Yahoo's raw payload instead.
Example output
{
"success": true,
"data": {
"query": "java developer",
"page": 1,
"page_size": 7,
"results": [
{
"url": "https://www.geeksforgeeks.org/gfg-academy/how-to-become-a-java-developer/",
"title": "How to become a Java Developer? - GeeksforGeeks",
"description": "What is a Java Developer? A Java Developer is a software engineer who builds, maintains, and improves applications using Java.",
"text": " * Careers & Jobs in Java\n\nCareers in Java are in high demand...\n",
"logo_url": "https://s.yimg.com/pv/static/search_favicon/images/32x32_7eae5aac8b7f7402.png",
"links": [
"https://www.geeksforgeeks.org/java/careers-jobs-in-java/"
],
"domain": "www.geeksforgeeks.org"
}
],
"next_cursor": "eyJxIjoiamF2YSBkZXZlbG9wZXIiLCJiIjo4LCJ0cCI6IkFueXRpbWUiLCJiZiI6dHJ1ZSwiYmMiOjgsImJsIjoxMCwiaCI6ImFiYzEyM2RlZiJ9"
}
}The catalog's sample response for queries=java developer, labelled as a sample: live calls return the account's current values.
Result
| Field | Type | Example | Description |
|---|---|---|---|
results.url | string | "https://www.geeksforgeeks.org/gfg-academy/h…" | Destination URL of the result |
results.title | string | "How to become a Java Developer? - GeeksforG…" | Result title as Yahoo shows it |
results.description | string | "What is a Java Developer? A Java Developer …" | Description text under the title |
results.text | string | " * Careers & Jobs in Java\n\nCareers in Ja…" | Extracted text of the result page |
results.logo_url | string | "https://s.yimg.com/pv/static/search_favicon…" | Favicon URL of the result's site |
results.links | array | 1 items | Sub-links found for the result |
results.domain | string | "www.geeksforgeeks.org" | Host of the result |
Response envelope and paging
| Field | Type | Example | Description |
|---|---|---|---|
success | boolean | true | true on every 200 response |
data.query | string | "java developer" | Your search term, echoed back |
data.page | integer | 1 | Page number of this response |
data.page_size | integer | 7 | Results per page |
data.next_cursor | string | "eyJxIjoiamF2YSBkZXZlbG9wZXIiLCJiIjo4LCJ0cCI…" | Pass back as cursor for the next page; null on the last page |
How does it work?
Scrapier sends each request through its residential proxy pool (use_proxy, on by default), so you never manage IPs or proxy accounts.
Retries and parsing of Yahoo's results pages happen on Scrapier's side. You send one HTTP request and get typed JSON back.
Only publicly visible data is returned: the results any visitor sees on Yahoo Search without signing in. No Yahoo account or cookie is used.
The schema is stable. When Yahoo changes its page, Scrapier updates the scraper; the field names under results stay the same.
How do you integrate Yahoo Search Scraper into your stack?
It works with what you already use: any HTTP client, AI agents over MCP, and no-code automation tools. Code samples for cURL, Python and Node.js are on the API tab.
MCP integration for AI agents
This registers the Scrapier MCP server, whose tools reach every Scrapier scraper, including yahoo-search: the agent finds it with search_endpoints and calls it with run_scrape. Works with Claude Code, Cursor and Windsurf. MCP calls are billed like REST: 1 credit per successful call. Full MCP setup →
No-code tools (n8n, Make, LangChain)
Use the HTTP Request node with your Scrapier API key in an X-API-Key header and point it at /api/v2/yahoo-search/ with a queries query parameter. Map data.results into the next node.
Add an HTTP "Make a request" module with the same header and URL, and parse the response as JSON. Map title, url and description straight into Sheets, Airtable or your CRM.
Wrap the endpoint in a requests-based tool, or load the Scrapier MCP server through LangChain's MCP adapters so the agent finds yahoo-search with search_endpoints itself.
Pricing and Rate Limits
One Yahoo page of results costs 1 credit, so 1,000 pages of results cost 1,000 credits: $1.58 on the Pro plan. Failed and empty calls are not billed. Run up to 3 requests in parallel per key; above that the API answers 429 with a Retry-After header.
| Plan | Monthly price | Credits | Pages of results included | Per 1,000 pages of results |
|---|---|---|---|---|
| Hobby | $29 | 15,000 | 15,000 | $1.93 |
| Pro | $79 | 50,000 | 50,000 | $1.58 |
| Scale | $299 | 250,000 | 250,000 | $1.20 |
| Enterprise | Custom volume and terms | |||
Prices from the live billing page. See all plans and limits.
Call cost and timing
How we measure this
Credit cost, pagination and the sample come from the live catalog (GET /api/v2/endpoints, /yahoo-search/, checked 9 Nov 2026). Concurrency and timeout guidance come from the rate limits page of the docs. Scrapier does not publish latency or success-rate figures for this endpoint, so none are shown here.
Yahoo Search Scraper vs the Official Yahoo API
| Feature | Scrapier Yahoo scraper | DIY Yahoo scraping |
|---|---|---|
| Access needed | One Scrapier API key | Your own proxies, parser and servers |
| Which data | Results for any query, as Yahoo shows them | Whatever your parser extracts |
| Fields returned | 7 fields per result: URL, title, description, text, logo, links and domain | Whatever you build and maintain |
| Setup | Sign up, copy a key, send a request | Write selectors, handle blocks and retries |
| Cost | 1 credit per page, from $1.20 per 1,000 pages | Proxy, server and engineering time |
If you need a fully custom crawler or already run your own scraping stack, DIY gives more control; Scrapier suits teams that want one key and one JSON shape across many platforms.
Is it legal to scrape Yahoo data?
Scraping publicly available search results is generally lawful in many countries, but the answer depends on what you collect, where you are and how you use it. In the US, hiQ Labs v. LinkedIn is the case most often cited on public-data scraping. Scrapier returns only data any visitor can see without logging in. Under GDPR and similar laws, collecting public data can be permissible, but how you store and use personal data must still comply with local data-protection rules, and Yahoo's terms apply. Consult legal counsel if your use case involves bulk storage of personal data. This is not legal advice; see our Terms of Use.
Frequently asked questions
What fields does Yahoo Search Scraper return?
Each result has url, title, description, text, logo_url, links and domain: 7 fields in data.results. The envelope adds query, page, page_size and next_cursor. Each call costs 1 credit.
Is there a Yahoo API for search results?
Scrapier's Yahoo scraper API is a REST endpoint at /api/v2/yahoo-search/: send a query and your key in the X-API-Key header, and get Yahoo results as JSON. It costs 1 credit per page and failed calls are free.
How do I filter Yahoo results by date?
Set time_period to Anytime, Past day, Past week or Past month. The default is Anytime. The value is locked into the cursor, so keep it the same while you page through one query's results.
How do I get the next page of Yahoo results?
Take data.next_cursor from the response and send it back unchanged as cursor. Repeat until next_cursor is null, which means you have reached the last page. Each page is one call and costs 1 credit.
Can I search several terms in one call?
No. queries takes one term per call. To search ten terms, make ten calls, up to 3 in parallel per API key, and page each with cursor. Throttled calls return 429 with a Retry-After header and cost nothing.
What do the backfill options do?
When Yahoo's snippet for a result is empty, backfill_empty_results (true by default) fetches that result's own page to fill in text and links. backfill_concurrency and backfill_max_links tune how many pages are fetched at once and how many sub-links are kept.
Does Yahoo Search Scraper require a Yahoo account or login?
No. The Yahoo Search Scraper uses no Yahoo account or cookies: you send your Scrapier API key in the X-API-Key header and a search term, and get JSON back. Only public results are returned.
What does raw_data do?
When raw_data is true, the response carries the raw upstream payload Scrapier received from Yahoo under raw_responses, in place of the formatted results. Paging with cursor and next_cursor works the same in both modes.
How fast does Yahoo Search Scraper run?
Scrapier does not publish a latency figure for this endpoint. Each call reads Yahoo live, and backfilling result pages can add time. Set a client timeout of 210 seconds, as the docs recommend for every endpoint.
What happens if the input is wrong or Yahoo returns nothing?
If a call fails, the API returns a non-2xx status and costs 0 credits. In code, check the status, then skip that input or retry it later. Throttled calls (429 with a Retry-After header) are free as well, so retry loops cost nothing.
Does Yahoo Search Scraper work with Claude, ChatGPT and other AI agent tools?
Yes. Connect the Scrapier MCP server with claude mcp add -t http scrapier https://mcp.scrapier.io/mcp and your API key in an X-API-Key header; it works in Claude Code, Cursor and Windsurf. The agent finds yahoo-search with search_endpoints and calls it with run_scrape. Any agent that can call an HTTP endpoint can use the REST API directly.
How much does it cost to run Yahoo Search Scraper 1,000 times?
1,000 credits, because each call is 1 credit. That is about $1.93 on Hobby, $1.58 on Pro and $1.20 on Scale, at the plans' price per credit. Failed calls are free.
How do I use Yahoo Search Scraper with Python?
For a Yahoo scraper in Python, send a GET request with the requests library to https://proxy.scrapier.io/api/v2/yahoo-search/ with params={"queries": "java developer"}, your key in the X-API-Key header and timeout=210. Read r.json()["data"]["results"]: it is already JSON, so no HTML parsing is needed.
Your feedback
Found a bug or missing a field? Let us know. Write to support@scrapier.io and the team replies to every report.
Start Scraping Yahoo Pages Of Results
Public Yahoo pages of results as JSON, 1 credit each, from REST or MCP.