Yahoo Search Scraper API
Last updated: · REST · JSON
Send GET https://proxy.scrapier.io/api/v2/yahoo-search/?queries=java developer with your key in the X-API-Key header. You get the page of results back as JSON. 1 credit per page of results; failed calls are free.
- 1 credit per page of results
- Failed calls free
- 3 parallel requests per key
Call the Yahoo Search Scraper API
Use the Yahoo Search Scraper API from your own apps, scripts or agents. You need a Scrapier account and the API key from your dashboard. Pick a language.
import requests# Replace YOUR_API_KEY with the key from your Scrapier dashboardAPI_KEY = "YOUR_API_KEY"params = { "queries": "java developer", "time_period": "Anytime", "backfill_empty_results": "true", "backfill_concurrency": "8", "backfill_max_links": "10",}r = requests.get( "https://proxy.scrapier.io/api/v2/yahoo-search/", headers={"X-API-Key": API_KEY}, params=params, timeout=210,)r.raise_for_status()# The result sits under data.resultsresult = r.json()["data"]["results"]print(result)
// Replace YOUR_API_KEY with the key from your Scrapier dashboardconst API_KEY = "YOUR_API_KEY";const params = new URLSearchParams({ queries: "java developer", time_period: "Anytime", backfill_empty_results: "true", backfill_concurrency: "8", backfill_max_links: "10",});const res = await fetch(`https://proxy.scrapier.io/api/v2/yahoo-search/?${params}`, { headers: { "X-API-Key": API_KEY }, signal: AbortSignal.timeout(210_000),});if (!res.ok) throw new Error(`HTTP ${res.status}`);const { data } = await res.json();console.log(data.results);
# One GET request. The key goes in a header, never in the URL.GET /api/v2/yahoo-search/?queries=java developer&time_period=Anytime&backfill_empty_results=true&backfill_concurrency=8&backfill_max_links=10 HTTP/1.1Host: proxy.scrapier.ioX-API-Key: YOUR_API_KEYAccept: application/json
# Scrapier publishes an OpenAPI spec that covers every endpoint. No API key needed.curl -O https://proxy.scrapier.io/api/v2/openapi.json# Import openapi.json into Postman, Insomnia, Bruno or a code generator,# then add your key as the X-API-Key header.
# 1. Add the Scrapier MCP server to Claude Codeclaude mcp add -t http scrapier https://mcp.scrapier.io/mcp --header "X-API-Key: YOUR_API_KEY"# 2. Ask your agent. It finds the endpoint with search_endpoints# and calls it with run_scrape:"Search Yahoo for java developer from the past week and list the top results"run_scrape("yahoo-search", {queries: "java developer"})
Yahoo Search Scraper API in Python
Plain requests is all you need — no SDK. Every response is typed JSON.
Install
Yahoo Search Scraper API in JavaScript
Works in Node 18+, Deno, Bun and edge runtimes with the built-in fetch.
Check your setup
Yahoo Search Scraper API over HTTP
One GET request. Auth is a single X-API-Key header.
Check your setup
Yahoo Search Scraper API OpenAPI definition
Download the spec once, then import it into the tools you already use. No key is needed to fetch it.
Download the spec
Yahoo Search Scraper API over MCP
The same endpoint is a tool call for Claude Code, Cursor and Windsurf. Same queries, same 1 credit per page of results.
Install
Other ways to call it
Yahoo Search Scraper API Interactive Playground
Edit the request, watch the code update, then run it. Without a key it shows the catalog sample and bills nothing. With your key it makes a real call from your browser; the key goes only to proxy.scrapier.io and is never stored.
GET /api/v2/yahoo-search/?queries=java developerPress Run request to send it. Without a key you get the catalog sample.
Failed calls are not billed.
The sample is the same for every input: it shows the response shape, not the account you typed.
Example response, catalog sample from 9 Nov 2026
{
"success": true,
"data": {
"query": "java developer",
"page": 1,
"page_size": 7,
"results": [
{
"url": "https://www.geeksforgeeks.org/gfg-academy/how-to-become-a-java-developer/",
"title": "How to become a Java Developer? - GeeksforGeeks",
"description": "What is a Java Developer? A Java Developer is a software engineer who builds, maintains, and improves applications using Java.",
"text": " * Careers & Jobs in Java\n\nCareers in Java are in high demand...\n",
"logo_url": "https://s.yimg.com/pv/static/search_favicon/images/32x32_7eae5aac8b7f7402.png",
"links": [
"https://www.geeksforgeeks.org/java/careers-jobs-in-java/"
],
"domain": "www.geeksforgeeks.org"
}
],
"next_cursor": "eyJxIjoiamF2YSBkZXZlbG9wZXIiLCJiIjo4LCJ0cCI6IkFueXRpbWUiLCJiZiI6dHJ1ZSwiYmMiOjgsImJsIjoxMCwiaCI6ImFiYzEyM2RlZiJ9"
}
}Data You Can Extract
Yahoo Search Scraper extracts the results page Yahoo returns for a search term, one page per call: each result's destination URL, title, description, extracted page text, favicon, sub-links and domain, plus the page number and the query it answered. Every field below comes from the catalog sample response, as typed JSON.
Result
7 fields in the response.
- url
- title
- description
- text
- logo_url
- links
- domain
Response envelope and paging
5 fields in the response.
- success
- data.query
- data.page
- data.page_size
- data.next_cursor
Response Schema Reference
Every field in the sample response. Field names stay the same on every call.
| Field | Type | Example | Description |
|---|---|---|---|
results.url | string | "https://www.geeksforgeeks.org/gfg-academy/h…" | Destination URL of the result |
results.title | string | "How to become a Java Developer? - GeeksforG…" | Result title as Yahoo shows it |
results.description | string | "What is a Java Developer? A Java Developer …" | Description text under the title |
results.text | string | " * Careers & Jobs in Java\n\nCareers in Ja…" | Extracted text of the result page |
results.logo_url | string | "https://s.yimg.com/pv/static/search_favicon…" | Favicon URL of the result's site |
results.links | array | 1 items | Sub-links found for the result |
results.domain | string | "www.geeksforgeeks.org" | Host of the result |
success | boolean | true | true on every 200 response |
data.query | string | "java developer" | Your search term, echoed back |
data.page | integer | 1 | Page number of this response |
data.page_size | integer | 7 | Results per page |
data.next_cursor | string | "eyJxIjoiamF2YSBkZXZlbG9wZXIiLCJiIjo4LCJ0cCI…" | Pass back as cursor for the next page; null on the last page |
Request Parameters Reference
| Name | Type | Required | Default | Description | Example |
|---|---|---|---|---|---|
queries | string | Yes | — | A search term. One per call. Required on the first call; ignored when cursor is supplied. Aliases: query, q | java developer |
cursor | string | No | — | Pass back data.next_cursor from the previous response, unchanged, to get the next page; null means the last page | — |
time_period | string | No | Anytime | Filter by recency, default Anytime. Options: Anytime, Past day, Past week, Past month. Locked in the cursor | Anytime |
backfill_empty_results | boolean | No | true | Fetch each result's own page to fill in text and links when Yahoo's snippet is empty, default true. Locked in the cursor | true |
backfill_concurrency | integer | No | 8 | Concurrent backfill page fetches per page, default 8. Locked in the cursor | 8 |
backfill_max_links | integer | No | 10 | Maximum sub-links to extract per backfilled result page, default 10. Locked in the cursor | 10 |
use_proxy | boolean | No | true | Route through Scrapier's residential proxy pool, on by default. Leave as is | true |
raw_data | boolean | No | false | Return Yahoo's raw upstream payload under raw_responses instead of the formatted items | true |
One term per call: to search ten terms, make ten calls and page each with cursor. Booleans are the strings true or false.
Pagination in the Yahoo Search Scraper API
The Yahoo Search Scraper API returns results in pages. Pass data.next_cursor back as cursor to get the next page; when it is null there are no more. Each successful call, first page or later, costs 1 credit.
Access Without Yahoo Login
The Yahoo Search Scraper API needs no Yahoo account, no login and no session cookies. You send your Scrapier key in the X-API-Key header and get JSON back. It reads public data only: what any visitor sees without logging in.
Speed, Cost and Concurrency
| Metric | Value | Source |
|---|---|---|
| Cost of a successful call | 1 credit | Scrapier catalog |
| Cost of a failed call | 0 credits | Any non-2xx response |
| Parallel requests per key | 3 | Above that: 429 with Retry-After |
| Recommended client timeout | 210 seconds | Scrapier docs, every endpoint |
Latency of the Yahoo Search Scraper API
Scrapier does not publish a latency or success-rate figure for this endpoint. Each call reads Yahoo at call time, so response time depends on Yahoo and on the size of the result. Set a client timeout of 210 seconds, as the docs recommend for every endpoint.
Limits of the Endpoint
Honest constraints of the Yahoo Search Scraper API.
- Public data only.
- A failed call, for example a missing or unavailable page of results, returns a non-2xx status and costs 0 credits.
- Data is read live at call time.
- 3 parallel requests per key; above that the API answers
429withRetry-After. - Large results come in pages; pass
data.next_cursorback ascursor. - Fields Yahoo does not show are not returned.
Scrapier vs the Official Yahoo API
If you need a fully custom crawler or already run your own scraping stack, DIY gives more control; Scrapier suits teams that want one key and one JSON shape across many platforms.
| Feature | Scrapier Yahoo scraper | DIY Yahoo scraping |
|---|---|---|
| Access needed | One Scrapier API key | Your own proxies, parser and servers |
| Which data | Results for any query, as Yahoo shows them | Whatever your parser extracts |
| Fields returned | 7 fields per result: URL, title, description, text, logo, links and domain | Whatever you build and maintain |
| Setup | Sign up, copy a key, send a request | Write selectors, handle blocks and retries |
| Cost | 1 credit per page, from $1.20 per 1,000 pages | Proxy, server and engineering time |
Errors and Retry Handling
| Status | Meaning | Billed | Fix |
|---|---|---|---|
| 200 | Page of results data returned in data.results | Yes | — |
| 400 | Bad or missing input | No | Check queries: one value per call |
| 401 | X-API-Key header missing | No | Add the header |
| 402 | Out of credits (body has balance) | No | Top up; retrying will not help |
| 403 | Invalid or revoked API key | No | Check the key in your dashboard |
| 429 | Throttled | No | Wait Retry-After seconds, then retry |
| 5xx | Upstream failure or timeout | No | Retry with backoff; set a 210 s client timeout |
From the error reference. Any non-2xx response costs 0 credits.
Retry logic for the Yahoo Search Scraper API
import time, requests def get_search(value, key, tries=5): for attempt in range(tries): r = requests.get( "https://proxy.scrapier.io/api/v2/yahoo-search/", headers={"X-API-Key": key}, params={"queries": value}, timeout=210, ) if r.status_code == 429: time.sleep(int(r.headers.get("Retry-After", 2 ** attempt))) elif r.status_code >= 500: time.sleep(2 ** attempt) else: r.raise_for_status() return r.json() raise RuntimeError("gave up after retries")
Pricing and Rate Limits
| Plan | Monthly price | Pages Of Results included | Cost per 1,000 |
|---|---|---|---|
| Hobby | $29 | 15,000 | $1.93 |
| Pro | $79 | 50,000 | $1.58 |
| Scale | $299 | 250,000 | $1.20 |
Billing rules for the Yahoo Search Scraper API
The Yahoo Search Scraper API bills 1 credit per successful page of results. Failed calls, meaning any non-2xx status, cost 0 credits. Every plan allows 3 parallel requests per key. The free tier lets you try it before you pay.
Bulk Lookups and CSV
queries takes one value per call. For a list, loop and run up to 3 requests in parallel per key, then flatten each row into one CSV line.
import csv, requests from concurrent.futures import ThreadPoolExecutor values = ["java developer"] # your list def fetch(value): r = requests.get( "https://proxy.scrapier.io/api/v2/yahoo-search/", headers={"X-API-Key": "YOUR_API_KEY"}, params={"queries": value}, timeout=210, ) r.raise_for_status() out = r.json()["data"]["results"] return out if isinstance(out, list) else [out] # 3 parallel requests per key is the limit with ThreadPoolExecutor(max_workers=3) as pool: rows = [row for part in pool.map(fetch, values) for row in part] cols = ["url", "title", "description", "domain", "logo_url", "links"] with open("out.csv", "w", newline="") as f: w = csv.DictWriter(f, fieldnames=cols, extrasaction="ignore") w.writeheader() w.writerows(rows)
CSV export from the Yahoo Search Scraper API
url,title,description,domain,logo_url,links https://www.geeksforgeeks.org/gfg-academy/how-to-become-a-java-developer/,How to become a Java Developer? - GeeksforGeeks,What is a Java Developer? A Java Developer is a software engineer who builds, maintains, and improves applications using Java.,www.geeksforgeeks.org,https://s.yimg.com/pv/static/search_favicon/images/32x32_7eae5aac8b7f7402.png,["https://www.geeksforgeeks.org/java/careers-jobs-in-java/"]
AI Agent Access via MCP
MCP for the Yahoo Search Scraper API
The same endpoint is available to AI agents through the Scrapier MCP server at https://mcp.scrapier.io/mcp. One command registers it in Claude Code: claude mcp add -t http scrapier https://mcp.scrapier.io/mcp --header "X-API-Key: YOUR_API_KEY". The agent finds the endpoint with search_endpoints and calls it with run_scrape and the endpoint id yahoo-search. It takes the same queries input and costs the same 1 credit per page of results. Setup for every client is on the Yahoo Search Scraper MCP page.
Integration Paths by Role
n8n and Make
- Add an HTTP Request step
- GET the endpoint with your
X-API-Keyheader - Map the
data.resultsfields to your columns
Loop and paginate
MCP tool in your agent
Call run_scrape with endpoint yahoo-search from Claude, Cursor or Windsurf. Setup is on the MCP page.
CSV and flattened JSON
Flatten each row into one CSV line, one column per field, ready for Sheets, Excel or a BI tool.
Legal and Compliance Notes
Scraping publicly available search results is generally lawful in many countries, but the answer depends on what you collect, where you are and how you use it. In the US, hiQ Labs v. LinkedIn is the case most often cited on public-data scraping. Scrapier returns only data any visitor can see without logging in. Under GDPR and similar laws, collecting public data can be permissible, but how you store and use personal data must still comply with local data-protection rules, and Yahoo's terms apply. Consult legal counsel if your use case involves bulk storage of personal data. This is not legal advice; see our Terms of Use.
Frequently Asked Questions
Is there a Yahoo page of results API?
Yes. Scrapier's Yahoo Search Scraper API returns any public page of results as JSON with one GET request and an X-API-Key header. No Yahoo app or login.
How do I call it from Python?
Use <code>requests.get</code> on <code>https://proxy.scrapier.io/api/v2/yahoo-search/</code> with <code>params={'queries': 'java developer'}</code> and <code>headers={"X-API-Key": KEY}</code>. Read <code>r.json()["data"]["results"]</code>.
What happens when a call fails?
Any non-200 response is free. Retry 429 and 5xx with backoff; the API already retries upstream failures.
How much does it cost?
1 credit per successful call — about $1.58 per 1,000 pages of results on the Pro plan. Failed calls are free.
Is there an OpenAPI spec?
Yes. It needs no key and covers every Scrapier endpoint.
Related Yahoo APIs
Fetch public Yahoo pages of results as typed JSON in one request
1 credit per page of results · failed calls free