Web Scraping 爬取网站 API
爬取一个网站，在一个标准化响应中返回最多 10 个页面的干净文本内容，适合把网站内容输入 LLM 和搜索索引。
US$1.25/千条结果，按请求付费，无需订阅。
Example - Build something great
Example is the platform teams use to ship faster…
{"url": "https://www.example.com","title": "Example - Build something great","text": "Example is the platform teams use to ship faster…","author": string | null,"canonicalUrl": string | null,"contentType": string | null,"description": string,"domain": string,... 还有 9 个字段}
价格与服务商
Web Scraping 爬取网站 API 价格：一个 API，3 个可互换的服务商
最便宜的服务商优先响应。如果它失败，下一个会在同一次调用中接手。
每次请求封顶 US$0.01251 条结果US$0.001255 条结果US$0.0062510 条结果US$0.0125（封顶）按实际返回的内容计费。调低返回数量上限即可少付，而且无论返回多少，单次请求都不会超过封顶价。2Bison备用 1可用率 59.50%p50 14389ms占调用的 63.7%63.7%72 次调用US$1.25/千条结果每次请求最多 US$0.0125计量方式：每条结果，按实际返回每条结果 US$0.00125
每次请求封顶 US$0.01251 条结果US$0.001255 条结果US$0.0062510 条结果US$0.0125（封顶）按实际返回的内容计费。调低返回数量上限即可少付，而且无论返回多少，单次请求都不会超过封顶价。3Salamander备用 2可用率 32.10%p50 14268ms占调用的 23.0%23.0%26 次调用US$1.25/千条结果每次请求最多 US$0.0125计量方式：每条结果，按实际返回每条结果 US$0.00125
每次请求封顶 US$0.01251 条结果US$0.001255 条结果US$0.0062510 条结果US$0.0125（封顶）按实际返回的内容计费。调低返回数量上限即可少付，而且无论返回多少，单次请求都不会超过封顶价。
试一试
一个请求获取 Web Scraping 爬取网站
这个示例展示了 31 个字段中的 6 个。查看全部字段
{
"found": true,
"data": {
"items": [
{
"url": "https://www.example.com",
"title": "Example - Build something great",
"text": "Example is the platform teams use to ship faster…"
},
{
"url": "https://www.example.com/pricing",
"title": "Pricing - Example",
"text": "Simple plans for teams of every size…"
}
]
}
}完整的参数与响应参考：这个接口的每个字段、类型和示例。
参考
一个请求，一种响应结构。
你发送的每个字段和收到的每个字段，都附有真实示例。
最近验证于 2026-11-14 · 可用率和延迟按 30d 统计
请求体
JSON，POST 到这个接口。
- urlstring
https://www.example.com
The website URL or domain to crawl. Pages linked from it on the same site are crawled too.
- limitinteger
Maximum number of pages to crawl and return (1-10, default 10). You are billed per page returned, so a lower limit costs less.
- requireFieldsarray
Optional; omit it and routing is unchanged. Name the output fields this request must be able to return, for example `language`, and it is served only by a source that returns every one of them. Fields you do not name are still returned whenever the serving source has them. Every source charges the same per page, so this never changes your price. A named field can still be empty on a page that genuinely lacks it.
响应
JSON，每个字段一个示例值。
Pages the crawl reported as failed or skipped. They are never in items and never billed. The list holds what the crawl reported, not every page of the site.
- outcome"failed" | "robots_disallowed" | "skipped"
failed: the page could not be fetched or answered with an HTTP error. robots_disallowed: the site's robots.txt does not allow crawling it. skipped: the crawl did not fetch it, for example because it is outside the crawled site.
- statusCodeinteger
The HTTP status code the page answered with, or null when unknown.
- urlstring
The page's URL.
One record per crawled page: its URL and domain, page title, meta description, language, HTTP status, the page content as Markdown text, and the page metadata the source reports. You are billed for these pages only. Populated whenever the provider has data for the entity.
- authorstring可设为必填
The author the page declares, or null when unknown.
- canonicalUrlstring可设为必填
The canonical URL the page declares in its own markup, or null when it declares none or the source does not report it.
- contentTypestring可设为必填
The content type the page was served with (for example text/html;charset=utf-8), or null when unknown.
- descriptionstring
The page's meta description, empty when the page has none.
- domainstring
The page's host name, without a leading www. Populated whenever the provider has data for the entity.
- jsonLdany[]可设为必填
The JSON-LD structured data blocks the page publishes (schema.org Organization, LocalBusiness, BreadcrumbList and so on), as parsed JSON, or null when the source does not report them. They are the publisher's own claims, not verified facts.
- languagestring可设为必填
The page's declared language code (for example en-US), or null when unknown.
- modifiedUtcinteger可设为必填
When the page says it was last modified, as a UTC epoch timestamp in seconds (Unix time), or null when unknown. Multiply by 1000 for a JS Date in milliseconds.
- publishedUtcinteger可设为必填
When the page says it was published, as a UTC epoch timestamp in seconds (Unix time), or null when unknown. Multiply by 1000 for a JS Date in milliseconds.
- robotsstring可设为必填
The page's robots meta directives (for example noindex, nofollow), or null when it declares none or the source does not report them.
- siteNamestring可设为必填
The site name the page declares (its og:site_name), or null when unknown.
- sourceUrlstring
The URL the crawler requested for this page, before any redirect, or null when unknown.
- statusCodeinteger
The HTTP status code the page answered with, or null when unknown.
- textstring
Example is the platform teams use to ship faster…
The page content as Markdown. Populated whenever the provider has data for the entity.
- titlestring
Example - Build something great
The page title, empty when the page has none (a PDF, for example).
- urlstring
https://www.example.com
The crawled page's URL, after any redirect. Populated whenever the provider has data for the entity.
The page's headings in document order, each with its level (1 for an h1) and text, or null when the source does not report them.
- levelinteger
The heading level, 1 to 6.
- textstring
The heading text.
Counts for this crawl. They describe the pages this crawl visited and reported, not the whole website, and are not added together into a total.
- pagesReturnedinteger
The number of pages in items, which is the number you are billed for.
- reportedFailedinteger
The number of pages the crawl reported as failed, or null when the source does not report a complete count.
- reportedSkippedinteger
The number of pages the crawl reported as skipped, including those robots.txt disallows, or null when the source does not report a complete count.
- foundboolean
true
- reason"not_found"
Present only when `found` is false, and says why there is no result. `not_found`: the source states the target does not exist, or returned nothing for it. A `found: false` answer is a successful call, not an error, and `costUsd` is what it actually cost.
常见问题
关于 Web Scraping 爬取网站 API
AnyAPI 的 Web Scraping 爬取网站 API 只需一次 POST 请求 /v1/run/web.crawl，就以统一格式的 JSON 返回 Web Scraping 数据。爬取一个网站，在一个标准化响应中返回最多 10 个页面的干净文本内容，适合把网站内容输入 LLM 和搜索索引。AnyAPI 会在 3 个数据源之间路由每个请求，某个数据源失败时自动切换。费用：每千条结果 US$1.25，单次请求最多 US$0.0125，以美元计价，无需订阅，也没有每月最低消费。过去 30 天，通过 AnyAPI 发起的 Web Scraping 爬取网站 API 调用中有 95.5% 成功，响应时间中位数为 10.4 秒，共测量 1,208 次调用。