> ## Documentation Index > Fetch the complete documentation index at: https://getanyapi.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Web Scrape > Scrape any web page and get its content back as clean Markdown (or HTML, or raw HTML) plus title and metadata. **Price:** \$0.70 per 1,000 requests (flat per request - same cost regardless of results returned). **Routing:** 5 independent lanes are available in execution order. A failed attempt uses the next lane only when the request's payment authorizes that lane's price; the normalized schema stays the same. **Catalog:** [Web Scrape pricing and uptime](https://getanyapi.com/api/web/scrape) - live USD price, lane routing, and measured 30-day uptime. [Every Web endpoint](https://getanyapi.com/api/web). ## OpenAPI ````yaml /openapi.json post /v1/run/web.scrape openapi: 3.1.0 info: contact: email: support@getanyapi.com description: >- Any API, one wallet, USD, no subscriptions. Each API is a single discovered operation: send the normalized input, get a normalized result. Prices and payment outcomes are reported in USD; each operation documents the settlement policy for its enabled payment rails. title: AnyAPI version: 1.0.0 x-guidance: >- Use each operation's published method and path with its normalized JSON input (see the operation requestBody). Before setting a client or tool timeout, GET /v1/apis/{sku} and inspect its trailing-30-day latency p50/p95/p99 and sample; p99 is an observation, not a maximum. To pay: send your AnyAPI key (Authorization: Bearer, billed from your USD wallet). For operations that advertise an inline rail, pay per call inline with x402 - call with no key, receive HTTP 402 with a PAYMENT-REQUIRED header, then retry with the PAYMENT-SIGNATURE header (base, no account needed); or pay per call inline with MPP (Machine Payments Protocol) - call with no key, receive HTTP 402 with a WWW-Authenticate: Payment challenge, then retry with the Authorization: Payment header (tempo, no account needed). x402 settles after execution; execution failure is reported as released; mpp settles before execution; execution failure is reported as charged_undelivered. servers: - url: https://api.getanyapi.com security: - bearerAuth: [] - apiKeyAuth: [] tags: - name: ahrefs x-group: Ahrefs - name: airbnb x-group: Airbnb - name: alibaba x-group: Alibaba - name: amazon x-group: Amazon - name: apollo x-group: Apollo - name: appstore x-group: Appstore - name: bluesky x-group: Bluesky - name: booking x-group: Booking - name: capterra x-group: Capterra - name: chatgpt x-group: Chatgpt - name: coinmarketcap x-group: Coinmarketcap - name: company-enrichment x-group: Company_enrichment - name: company-search x-group: Company_search - name: congress x-group: Congress - name: dexscreener x-group: Dexscreener - name: douyin x-group: Douyin - name: ebay x-group: Ebay - name: email x-group: Email - name: email-finding x-group: Email_finding - name: email-verification x-group: Email_verification - name: facebook x-group: Facebook - name: fiverr x-group: Fiverr - name: g2 x-group: G2 - name: gemini x-group: Gemini - name: github x-group: Github - name: glassdoor x-group: Glassdoor - name: google x-group: Google - name: google-ads x-group: Google Ads - name: google-finance x-group: Google Finance - name: google-shopping x-group: Google Shopping - name: hackernews x-group: Hackernews - name: indeed x-group: Indeed - name: instagram x-group: Instagram - name: linkedin x-group: Linkedin - name: maps x-group: Maps - name: mobile-phone x-group: Mobile_phone - name: naver x-group: Naver - name: pandaexpress x-group: Pandaexpress - name: people-search x-group: People_search - name: perplexity x-group: Perplexity - name: person x-group: Person - name: person-enrichment x-group: Person_enrichment - name: pinterest x-group: Pinterest - name: playstore x-group: Playstore - name: polymarket x-group: Polymarket - name: realtor x-group: Realtor - name: reddit x-group: Reddit - name: redfin x-group: Redfin - name: rednote x-group: Rednote - name: sec x-group: Sec - name: semrush x-group: Semrush - name: seo x-group: Seo - name: snapchat x-group: Snapchat - name: social x-group: Social - name: spotify x-group: Spotify - name: substack x-group: Substack - name: threads x-group: Threads - name: tiktok x-group: TikTok - name: tiktok-shop x-group: TikTok Shop - name: tripadvisor x-group: Tripadvisor - name: trustpilot x-group: Trustpilot - name: truthsocial x-group: Truthsocial - name: twitter x-group: X (Twitter) - name: upwork x-group: Upwork - name: walmart x-group: Walmart - name: web x-group: Web - name: weibo x-group: Weibo - name: yahoo-finance x-group: Yahoo Finance - name: yelp x-group: Yelp - name: youtube x-group: YouTube - name: zhihu x-group: Zhihu - name: zillow x-group: Zillow paths: /v1/run/web.scrape: post: tags: - web summary: Web Scrape description: >- Scrape any web page and get its content back as clean Markdown (or HTML, or raw HTML) plus title and metadata. **Price:** \$0.70 per 1,000 requests (flat per request - same cost regardless of results returned). **Routing:** 5 independent lanes are available in execution order. A failed attempt uses the next lane only when the request's payment authorizes that lane's price; the normalized schema stays the same. **Catalog:** [Web Scrape pricing and uptime](https://getanyapi.com/api/web/scrape) - live USD price, lane routing, and measured 30-day uptime. [Every Web endpoint](https://getanyapi.com/api/web). operationId: web_scrape parameters: - description: >- Optional. Comma-separated keys (dotted paths like `author.name` descend into nested objects) to keep on each result item. Keys are matched relative to each result item after the `data`/`items` envelope is unwrapped, not against the top-level response envelope, so use `jq` to reshape the whole envelope. Shrinks the response without changing cost. in: query name: fields required: false schema: type: string - description: >- Optional. Cap the number of result rows returned; a `_truncated` note reports how many were withheld so you can page via the API's own `limit`. Does not change cost. in: query name: max_items required: false schema: minimum: 0 type: integer - description: >- Optional. Return only a structural outline (top-level keys, item counts, and per-field byte sizes) instead of the full data. Does not change cost. in: query name: summary required: false schema: type: boolean - description: >- Optional. A jq expression applied to the result envelope; its output replaces `output` (multiple outputs collect into an array). Reshape freely, e.g. `jq=.data | {title, description, md: .markdown[:3500]}`. Runs sandboxed with a 250ms / 2MB budget; on failure the full result is returned with a `jqError`. Does not change cost. in: query name: jq required: false schema: type: string - description: >- Optional wallet idempotency key, scoped to this customer for 24 hours. When the gateway honors the key, this synchronous in-process execution can continue after the caller disconnects, bounded by its execution deadline. A completed replayable result charges normally exactly once and can be replayed without another provider run or charge. A pending duplicate returns 409 idempotency_in_progress; reuse with different request semantics returns 409 idempotency_conflict. in: header name: Idempotency-Key required: false schema: maxLength: 255 minLength: 1 type: string requestBody: content: application/json: example: formats: - markdown - rawHtml onlyMainContent: false url: https://example.com schema: additionalProperties: false example: formats: - markdown - rawHtml onlyMainContent: false url: https://example.com properties: blockAds: description: >- When true (upstream default), strip ad and cookie-consent elements before capture. Set false to keep them. type: boolean excludeTags: description: >- CSS selectors to drop before capture (for example ["nav", "footer", ".ads"]). Applied after includeTags. items: type: string type: array formats: default: - markdown - rawHtml description: >- Which representations of the page to return. Any combination of: markdown (page content as Markdown), html (cleaned HTML), rawHtml (verbatim page HTML). Each requested format is returned under the matching output field. Defaults to Markdown and raw HTML. items: enum: - markdown - html - rawHtml type: string type: array includeTags: description: >- CSS selectors to keep. When set, only content matching these selectors is captured (for example ["article", "main"] or ["#content"]). items: type: string type: array mobile: description: >- When true, render the page with a mobile viewport and user agent instead of desktop. Some sites serve materially different content to mobile. type: boolean onlyMainContent: default: false description: >- When true, return only the main article content, stripping navigation, headers, footers, and other boilerplate. Defaults to false to capture the full page. type: boolean stealth: default: false description: >- When true, fetch through a stealth proxy that gets past bot protection which refuses an ordinary request. This costs materially more per request (see the pricing ceiling), so leave it off unless a normal scrape of the site comes back blocked or empty. type: boolean url: description: The URL of the page to scrape. format: uri minLength: 1 type: string waitFor: description: >- Milliseconds to wait for the page to finish rendering before capture. Use this for JavaScript-heavy pages or single-page apps whose content loads after the initial paint. Capped at 15000 to stay within the request timeout. This wait is time you asked us to spend, so your response takes this much longer, and it is excluded from the latency published for this endpoint. maximum: 15000 minimum: 0 type: integer x-anyapi-caller-delay: unit: ms required: - url title: Web scrape input type: object required: true responses: '200': content: application/json: schema: properties: costUsd: description: >- USD charged on the original run. On a replay this value is echoed for parity; the replay itself is free. type: number hint: description: >- Optional one-line nudge, absent when there is nothing to say. `large_result:` suggests the fields/max_items/summary/jq controls for a big response. `paging_unavailable:` means this result came from a source that cannot return a `nextCursor`, so it may be INCOMPLETE and cannot be continued - re-run with `requireCursor: true` to be served only by a source that can page, which may cost more per request. type: string items: description: >- Number of result rows returned. For per-result SKUs the per-item cost is charged against this count; for input-priced SKUs the charge is per submitted input, independent of this count. type: integer jqError: description: >- Present only when a `jq` expression failed; `output` then carries the full unshaped result and this explains why the reshape did not apply. type: string output: anyOf: - additionalProperties: false properties: data: oneOf: - type: 'null' - properties: description: description: The page meta description. type: string html: description: >- The cleaned page HTML. Present only when 'html' is among the requested formats. type: string markdown: description: >- The page content as clean Markdown. Present when 'markdown' is among the requested formats (the default). Populated whenever the provider has data for the entity. type: string x-anyapi-must-populate: true rawHtml: description: >- The verbatim page HTML before cleaning. Present only when 'rawHtml' is among the requested formats. type: string title: description: The page title from its metadata. type: string url: description: >- The canonical source URL of the scraped page. Populated whenever the provider has data for the entity. type: string x-anyapi-must-populate: true required: - url - title - description - markdown type: object found: type: boolean required: - found - data title: Web scrape output type: object x-anyapi-heavy: true - type: 'null' description: >- Normalized output, or null when the replay payload was not retained. provider: description: Always "AnyAPI". type: string replayed: description: >- True when this response replays the durable result of an earlier run without billing or upstream execution. type: boolean resultId: description: >- Opaque handle to the full unshaped result, cached ~15 min. Re-shape it for free (fields/max_items/summary/jq) via GET /v1/results/{id}, no re-billing. Absent when the result was too large to cache. type: string required: - output - provider - costUsd - items - replayed type: object description: Normalized result. headers: Idempotency-Replayed: description: >- Present only when the request carried an Idempotency-Key this gateway honored. "false" means the key was claimed and the run executed freshly; "true" means it replayed a prior identical run and was NOT charged again. Absent means either no key was sent or this gateway does not honor keys, so a retry could be charged twice. schema: enum: - 'false' - 'true' type: string PAYMENT-RESPONSE: description: Inline x402 payment receipt. schema: type: string Payment-Receipt: description: Inline mpp payment receipt. schema: type: string X-PAYMENT-RESPONSE: description: Inline x402 payment receipt. schema: type: string '400': content: application/json: schema: $ref: '#/components/schemas/Error' description: >- Input did not match the schema, or the upstream rejected the request. '401': content: application/json: schema: $ref: '#/components/schemas/Error' description: Missing or invalid API key. '402': content: application/json: schema: oneOf: - $ref: '#/components/schemas/PaymentRequiredError' - $ref: '#/components/schemas/TrialCapReached' application/problem+json: schema: properties: challengeId: minLength: 1 type: string status: const: 402 type: integer required: - status - challengeId type: object description: >- Insufficient wallet balance or spend cap, or an inline-payment challenge. Agent trials receive a trial_cap_reached continuation with a live RFC 8628 device authorization. headers: PAYMENT-REQUIRED: description: Inline x402 payment challenge. schema: type: string WWW-Authenticate: description: Inline mpp payment challenge. schema: type: string '404': content: application/json: schema: $ref: '#/components/schemas/Error' description: Unknown SKU, or the upstream had no matching resource. '409': content: application/json: schema: $ref: '#/components/schemas/Error' description: >- Idempotency collision. Codes: idempotency_in_progress means retry after the Retry-After delay; idempotency_conflict means use a new key for different request semantics; idempotency_needs_review means stop retrying and contact AnyAPI support with X-Anyapi-Request-Id. '422': content: application/json: schema: $ref: '#/components/schemas/Error' description: >- The request was well formed but this API cannot serve the requested target site; the body's `alternatives` array lists the AnyAPI APIs that can. '429': content: application/json: schema: $ref: '#/components/schemas/Error' description: >- The upstream was rate limited; inspect `payment` when present before deciding whether to retry. '500': content: application/json: schema: $ref: '#/components/schemas/Error' description: >- An internal error prevented completion. Inspect payment when present before retrying. '502': content: application/json: schema: $ref: '#/components/schemas/Error' description: >- Every provider failed with a server/transport error, or inline payment settlement could not be confirmed; inspect `payment` when present before retrying. security: - bearerAuth: [] - apiKeyAuth: [] - {} components: schemas: Error: properties: code: description: Stable machine-readable error code when the endpoint defines one. type: string error: description: Customer-safe error message. type: string payment: $ref: '#/components/schemas/Payment' required: - error type: object PaymentRequiredError: allOf: - $ref: '#/components/schemas/Error' - properties: error: not: const: trial_cap_reached type: object TrialCapReached: properties: error: const: trial_cap_reached type: string message: description: >- Human-relayable continuation instructions, including an executable token poll. type: string upgrade: $ref: '#/components/schemas/AgentUpgrade' required: - error - message - upgrade type: object Payment: properties: costUsd: description: Known USD payment amount for this request. minimum: 0 type: number rail: minLength: 1 type: string settlementState: enum: - charged_undelivered - indeterminate type: string required: - rail - settlementState - costUsd type: object AgentUpgrade: properties: authorizationServer: format: uri type: string clientId: type: string device: $ref: '#/components/schemas/DeviceUpgrade' scope: type: string required: - authorizationServer - scope type: object DeviceUpgrade: properties: deviceCode: type: string expiresIn: minimum: 0 type: integer grantType: type: string interval: minimum: 1 type: integer tokenEndpoint: format: uri type: string userCode: type: string verificationUri: format: uri type: string verificationUriComplete: format: uri type: string required: - verificationUriComplete - verificationUri - userCode - deviceCode - tokenEndpoint - grantType - interval - expiresIn type: object securitySchemes: bearerAuth: description: Your AnyAPI key as a Bearer token. scheme: bearer type: http x-default: YOUR_ANYAPI_KEY apiKeyAuth: description: Your AnyAPI key. in: header name: X-API-Key type: apiKey x-default: YOUR_ANYAPI_KEY ````