> ## Documentation Index
> Fetch the complete documentation index at: https://docs.context.dev/llms.txt
> Use this file to discover all available pages before exploring further.
# Scrape a URL
> Fetch a page once and return any combination of Markdown, HTML, a screenshot, images, original bytes, and CSS-selected fields.
1 Credit With actions: 2 Credits PDF OCR: +1 Credit per recovered page
Set `formats` to choose outputs. Each requested output returns `requested: true` with its `data`; outputs you did not request return `data: null`. One credit covers every format in the request, including cache hits. See [Scrape a webpage](/guides/scrape-websites-to-markdown) for content controls, freshness, and dynamic pages, or the format guides for [screenshots](/guides/take-webpage-screenshot), [images](/guides/extract-page-images), and [bytes](/guides/download-resource-bytes).
## OpenAPI
````yaml POST /web/scrape
openapi: 3.1.0
info:
title: Context.dev API
description: API for retrieving context data from any website
version: 1.0.0
servers:
- url: https://api.context.dev/v1
security: []
tags:
- name: Webhooks
description: Inspect and retry webhook deliveries. These endpoints cost no credits.
- name: Logs
description: >-
Read your organization's API request logs to debug failed calls. These
endpoints cost no credits and use a separate rate limit.
- name: Batch
description: Scrape many pages or crawl a site asynchronously.
- name: Monitors
description: >-
Monitor pages, sitemaps, and extracted website data for exact or semantic
changes. Webhook payloads are documented by the
MonitorsChangeDetectedWebhookPayload and
MonitorsRunCompletedWebhookPayload schemas.
- name: News
description: >-
Search live first-party RSS and free historical news data by company
identity.
- name: Answers
description: Answer a research task from the live web in the JSON shape you ask for.
paths:
/web/scrape:
post:
tags:
- Web Scraping
summary: Scrape
description: >-
Reuse cached outputs independently and capture missing formats in one
page visit. Each cache key includes only the settings that affect that
output. HTML is shared with Markdown and parsed fields. Cached outputs
can come from different visits within maxAgeMs; use 0 for a fresh
capture. HTML-only requests use the existing fast acquisition path. One
credit per request, including cache hits, or two with browser actions;
PDF OCR adds one credit per recovered page on fresh extraction. Original
response bytes and screenshots are limited to 20 MiB each, screenshots
to 40 megapixels, and the combined browser capture to 60 MiB.
operationId: scrapeWeb
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/ScrapeRequest'
examples:
Default:
summary: Get HTML
value:
url: https://example.com
formats:
html: true
Screenshot:
summary: Take a screenshot
value:
url: https://example.com
formats:
screenshot: true
Parse fields:
summary: Parse fields
value:
url: https://example.com
formats:
parse: true
parseParams:
rules:
title: h1
All outputs:
summary: All six outputs
value:
url: https://example.com
formats:
html: true
markdown: true
screenshot: true
images: true
bytes: true
parse: true
parseParams:
rules:
title: h1
With options:
summary: Shared settings, specific params, and request controls
value:
url: https://example.com
formats:
markdown: true
screenshot: true
images: true
sharedParams:
theme: dark
waitFor: main
mainContentOnly: true
excludeSelectors:
- nav
- footer
markdownParams:
includeLinks: false
screenshotParams:
area: fullPage
imageParams:
enrich:
- dimensions
maxAgeMs: 0
timeoutOpts:
milliseconds: 60000
behavior: fail
tags:
- product-sync
Shared inputs:
summary: Country, headers, waiting, and freshness
value:
url: https://example.com
formats:
screenshot: true
images: true
bytes: true
sharedParams:
country: US
headers:
Accept-Language: en-US
waitFor: 500
maxAgeMs: 0
responses:
'200':
description: >-
All six outputs are present. Unrequested outputs have requested:
false and data: null. Every requested output must succeed.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ScrapeResponse'
examples:
Default:
summary: HTML
value:
url: https://example.com/
html:
requested: true
data: Example Domain
markdown:
requested: false
data: null
screenshot:
requested: false
data: null
images:
requested: false
data: null
bytes:
requested: false
data: null
parsed:
requested: false
data: null
Screenshot:
summary: Screenshot
value:
url: https://example.com/
html:
requested: false
data: null
markdown:
requested: false
data: null
screenshot:
requested: true
data: >-
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9ZQmcAAAAASUVORK5CYII=
images:
requested: false
data: null
bytes:
requested: false
data: null
parsed:
requested: false
data: null
All outputs:
summary: All six outputs
value:
url: https://example.com/
html:
requested: true
data: Example Domain
markdown:
requested: true
data: '# Example Domain'
screenshot:
requested: true
data: >-
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9ZQmcAAAAASUVORK5CYII=
images:
requested: true
data:
- url: https://example.com/hero.jpg
alt: Product photo
bytes:
requested: true
data:
contentType: text/html
base64: PGgxPkV4YW1wbGUgRG9tYWluPC9oMT4=
parsed:
requested: true
data:
title: Example Domain
Parse fields:
summary: Parsed fields without an HTML output
value:
url: https://example.com/
html:
requested: false
data: null
markdown:
requested: false
data: null
screenshot:
requested: false
data: null
images:
requested: false
data: null
bytes:
requested: false
data: null
parsed:
requested: true
data:
title: Example Domain
Shared inputs:
summary: Shared browser settings
value:
url: https://example.com/
html:
requested: false
data: null
markdown:
requested: false
data: null
screenshot:
requested: true
data: >-
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9ZQmcAAAAASUVORK5CYII=
images:
requested: true
data:
- url: https://example.com/hero.jpg
alt: Product photo
bytes:
requested: true
data:
contentType: text/html; charset=utf-8
base64: >-
PGh0bWw+PGJvZHk+PGgxPkV4YW1wbGUgRG9tYWluPC9oMT48L2JvZHk+PC9odG1sPg==
parsed:
requested: false
data: null
'400':
description: >-
Invalid input, inaccessible content, or failed document parsing.
Unknown options and incompatible settings are rejected before
scraping.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'401':
description: Missing or invalid API key.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'403':
description: Zero data retention is not enabled for your organization.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'404':
description: The target resource was not found.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'408':
description: >-
The request exceeded timeoutOpts.milliseconds or no usable partial
result was available.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'413':
description: A response or screenshot exceeds the capture limits.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'415':
description: The resource does not support a requested output.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'422':
description: A browser action, selector, or requested output failed.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'429':
description: Rate limit exceeded.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'500':
description: The scrape could not be completed.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'502':
description: The browser could not return a complete capture.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'503':
description: Browser capacity is temporarily unavailable.
headers:
X-Request-Id:
$ref: '#/components/headers/RequestId'
X-RateLimit-Limit:
$ref: '#/components/headers/RateLimitLimit'
X-RateLimit-Remaining:
$ref: '#/components/headers/RateLimitRemaining'
X-RateLimit-Reset:
$ref: '#/components/headers/RateLimitReset'
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
security:
- bearerAuth: []
x-codeSamples:
- lang: json
label: JSON
source: |-
{
"url": "https://example.com",
"formats": {
"html": true
}
}
- lang: bash
label: cURL
source: |-
curl https://api.context.dev/v1/web/scrape \
-H "Authorization: Bearer $CONTEXT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com",
"formats": {
"html": true
}
}'
components:
schemas:
ScrapeRequest:
type: object
additionalProperties: false
properties:
url:
type: string
format: uri
pattern: ^https?://
description: The URL to scrape.
formats:
$ref: '#/components/schemas/ScrapeFormats'
description: Outputs to return. Enable at least one; omitted formats are false.
sharedParams:
$ref: '#/components/schemas/ScrapeSharedParams'
description: >-
Shared browser and content settings. Content filters leave
screenshots and original bytes unchanged.
markdownParams:
$ref: '#/components/schemas/ScrapeMarkdownParams'
description: 'Markdown options. Requires formats.markdown: true.'
screenshotParams:
$ref: '#/components/schemas/ScrapeScreenshotParams'
description: 'Screenshot options. Requires formats.screenshot: true.'
imageParams:
$ref: '#/components/schemas/ScrapeImageParams'
description: 'Image options. Requires formats.images: true.'
parseParams:
$ref: '#/components/schemas/ScrapeParseParams'
description: Required when formats.parse is true.
maxAgeMs:
type: integer
minimum: 0
maximum: 2592000000
default: 86400000
description: >-
Maximum age of each cached output. Defaults to 1 day; 0 fetches
fresh and updates the requested outputs. Compatible outputs are
shared with the individual scrape endpoints. Image results with
hosted files refresh after 23 hours; other outputs retain their own
freshness.
zdr:
type: string
enum:
- enabled
- disabled
default: disabled
description: >-
Zero data retention. Bypasses caches and uploads; excludes
request/response content and tags from logs. Must be enabled for
your organization.
timeoutOpts:
$ref: '#/components/schemas/ScrapeTimeout'
default:
milliseconds: 60000
behavior: fail
description: >-
Total deadline, including navigation, actions, waiting, and all
outputs. Defaults to 60000 milliseconds with behavior fail. Use
return-partial to capture the current page state and return captured
images if image processing cannot finish before the deadline; these
responses set isPartial and are not cached. Every requested format
must still be available. Fixed waits must fit before a response
reserve of up to 5000 milliseconds (at most one quarter of the
timeout) when using return-partial.
tags:
$ref: '#/components/schemas/RequestTags'
description: Labels for tracking request usage. Not retained when zdr is enabled.
required:
- url
- formats
allOf:
- if:
required:
- markdownParams
then:
properties:
formats:
required:
- markdown
properties:
markdown:
const: true
- if:
required:
- screenshotParams
then:
properties:
formats:
required:
- screenshot
properties:
screenshot:
const: true
- if:
required:
- imageParams
then:
properties:
formats:
required:
- images
properties:
images:
const: true
- if:
required:
- parseParams
then:
properties:
formats:
required:
- parse
properties:
parse:
const: true
- if:
properties:
formats:
required:
- parse
properties:
parse:
const: true
then:
required:
- parseParams
ScrapeResponse:
type: object
additionalProperties: false
properties:
isPartial:
type: boolean
const: true
description: >-
Present when return-partial captures a page that is still loading or
returns images before image processing finishes. Partial responses
are not cached.
url:
type: string
format: uri
pattern: ^https?://
description: Final URL after redirects and browser actions.
html:
type: object
additionalProperties: false
properties:
requested:
type: boolean
data:
type:
- string
- 'null'
required:
- requested
- data
if:
properties:
requested:
const: true
then:
properties:
data:
type: string
else:
properties:
data:
type: 'null'
description: Rendered HTML after content filters.
markdown:
type: object
additionalProperties: false
properties:
requested:
type: boolean
data:
type:
- string
- 'null'
required:
- requested
- data
if:
properties:
requested:
const: true
then:
properties:
data:
type: string
else:
properties:
data:
type: 'null'
description: Markdown after content filters.
screenshot:
type: object
additionalProperties: false
properties:
requested:
type: boolean
data:
type:
- string
- 'null'
format: uri
pattern: ^data:image/(?:png|jpeg|webp);base64,[A-Za-z0-9+/]+={0,2}$
required:
- requested
- data
if:
properties:
requested:
const: true
then:
properties:
data:
type: string
format: uri
pattern: ^data:image/(?:png|jpeg|webp);base64,[A-Za-z0-9+/]+={0,2}$
else:
properties:
data:
type: 'null'
description: An image data URL. Use directly as an image src.
images:
type: object
additionalProperties: false
properties:
requested:
type: boolean
data:
type:
- array
- 'null'
items:
$ref: '#/components/schemas/ScrapeImage'
required:
- requested
- data
if:
properties:
requested:
const: true
then:
properties:
data:
type: array
items:
$ref: '#/components/schemas/ScrapeImage'
else:
properties:
data:
type: 'null'
description: Images after content filters. Empty when none are found.
bytes:
type: object
additionalProperties: false
properties:
requested:
type: boolean
data:
type:
- object
- 'null'
additionalProperties: false
properties:
contentType:
type: string
examples:
- text/html; charset=utf-8
base64:
type: string
contentEncoding: base64
pattern: >-
^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$
description: >-
Original response body as base64, after HTTP decompression.
Maximum decoded size: 20 MiB.
required:
- contentType
- base64
required:
- requested
- data
if:
properties:
requested:
const: true
then:
properties:
data:
type: object
additionalProperties: false
properties:
contentType:
type: string
examples:
- text/html; charset=utf-8
base64:
type: string
contentEncoding: base64
pattern: >-
^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$
description: >-
Original response body as base64, after HTTP
decompression. Maximum decoded size: 20 MiB.
required:
- contentType
- base64
else:
properties:
data:
type: 'null'
description: >-
Original HTTP response body. Waiting, actions, and content filters
never change it.
parsed:
type: object
additionalProperties: false
properties:
requested:
type: boolean
data:
type:
- object
- 'null'
additionalProperties: true
required:
- requested
- data
if:
properties:
requested:
const: true
then:
properties:
data:
type: object
additionalProperties: true
else:
properties:
data:
type: 'null'
description: Fields produced by parseParams.rules, after shared content filters.
metadata:
$ref: '#/components/schemas/ScrapeMetadata'
description: Page details, when available.
cache_metadata:
$ref: '#/components/schemas/CacheMetadata'
request_id:
$ref: '#/components/schemas/RequestId'
key_metadata:
$ref: '#/components/schemas/KeyMetadata'
required:
- url
- html
- markdown
- screenshot
- images
- bytes
- parsed
- metadata
- cache_metadata
- request_id
not:
properties:
html:
properties:
requested:
const: false
markdown:
properties:
requested:
const: false
screenshot:
properties:
requested:
const: false
images:
properties:
requested:
const: false
bytes:
properties:
requested:
const: false
parsed:
properties:
requested:
const: false
ErrorResponse:
type: object
properties:
message:
type: string
description: Error message
error_code:
type: string
enum:
- INTERNAL_ERROR
- VALID
- NOT_FOUND
- FORBIDDEN
- USAGE_EXCEEDED
- RATE_LIMITED
- UNAUTHORIZED
- DISABLED
- PAID_PLAN_REQUIRED
- INSUFFICIENT_PERMISSIONS
- TIMEOUT_EXCEEDS_MAXIMUM
- TIMEOUT_TOO_SHORT_FOR_WAIT
- WEBSITE_ACCESS_ERROR
- WEBSITE_BLOCKED
- WEBSITE_NOT_FOUND
- PDF_SKIPPED
- PDF_IMAGES_ONLY
- EXTERNAL_PROVIDER_ERROR
- INPUT_VALIDATION_ERROR
- ZDR_NOT_SUPPORTED
- ZDR_NOT_ENABLED
- FREE_EMAIL_DETECTED
- DISPOSABLE_EMAIL_DETECTED
- REQUEST_TIMEOUT
- COLD_DOMAIN_TIMEOUT_TOO_LOW
- UNSUPPORTED_CONTENT
- CONTENT_TOO_LARGE
- MONITOR_PAUSED
- MONITOR_NO_WEBHOOK
- COLLECTION_PAUSED
- MONITOR_LIMIT_EXCEEDED
- SEARCH_UNAVAILABLE
- BATCH_LIMIT_EXCEEDED
- BATCH_NOT_CANCELLABLE
- BATCH_NOT_COMPLETED
- IDEMPOTENCY_KEY_CONFLICT
- DELIVERY_IN_PROGRESS
- DELIVERY_ALREADY_DELIVERED
- DELIVERY_EXPIRED
- DELIVERY_CANCELLED
description: Error code indicating the type of error
required_permission:
type: string
enum:
- logs:read
- data:execute
- monitors:read
- monitors:write
- batches:read
- batches:write
- webdbs:read
- webdbs:write
description: >-
Permission required for this request when error_code is
INSUFFICIENT_PERMISSIONS. Manage (write) also grants read access to
the same resource group.
key_metadata:
$ref: '#/components/schemas/KeyMetadata'
request_id:
$ref: '#/components/schemas/RequestId'
required:
- request_id
ScrapeFormats:
type: object
additionalProperties: false
properties:
html:
type: boolean
default: false
description: Rendered HTML.
markdown:
type: boolean
default: false
description: Page content as Markdown.
screenshot:
type: boolean
default: false
description: An inline image of the page.
images:
type: boolean
default: false
description: Images found on the page.
bytes:
type: boolean
default: false
description: The original HTTP response body.
parse:
type: boolean
default: false
description: Fields selected by parseParams.rules.
not:
allOf:
- not:
required:
- html
properties:
html:
const: true
- not:
required:
- markdown
properties:
markdown:
const: true
- not:
required:
- screenshot
properties:
screenshot:
const: true
- not:
required:
- images
properties:
images:
const: true
- not:
required:
- bytes
properties:
bytes:
const: true
- not:
required:
- parse
properties:
parse:
const: true
ScrapeSharedParams:
type: object
additionalProperties: false
properties:
country:
type: string
pattern: ^[A-Za-z]{2}$
examples:
- US
description: >-
Supported two-letter country code, case-insensitive. Applies to
every output, including image downloads.
headers:
type: object
propertyNames:
pattern: ^[!#$%&'*+.^_`|~0-9A-Za-z-]+$
not:
pattern: >-
^(?:[Hh][Oo][Ss][Tt]|[Cc][Oo][Nn][Tt][Ee][Nn][Tt]-[Ll][Ee][Nn][Gg][Tt][Hh]|[Cc][Oo][Nn][Nn][Ee][Cc][Tt][Ii][Oo][Nn]|[Kk][Ee][Ee][Pp]-[Aa][Ll][Ii][Vv][Ee]|[Pp][Rr][Oo][Xx][Yy]-[Aa][Uu][Tt][Hh][Ee][Nn][Tt][Ii][Cc][Aa][Tt][Ee]|[Pp][Rr][Oo][Xx][Yy]-[Aa][Uu][Tt][Hh][Oo][Rr][Ii][Zz][Aa][Tt][Ii][Oo][Nn]|[Pp][Rr][Oo][Xx][Yy]-[Cc][Oo][Nn][Nn][Ee][Cc][Tt][Ii][Oo][Nn]|[Tt][Ee]|[Tt][Rr][Aa][Ii][Ll][Ee][Rr]|[Tt][Rr][Aa][Nn][Ss][Ff][Ee][Rr]-[Ee][Nn][Cc][Oo][Dd][Ii][Nn][Gg]|[Uu][Pp][Gg][Rr][Aa][Dd][Ee]|[Ee][Xx][Pp][Ee][Cc][Tt])$
additionalProperties:
type: string
maxLength: 8192
pattern: ^[^\r\n]*$
description: >-
Headers for the target origin. Requests with custom headers bypass
caching.
examples:
- Accept-Language: en-US
viewport:
type: object
additionalProperties: false
properties:
width:
type: integer
minimum: 240
maximum: 7680
default: 1440
height:
type: integer
minimum: 240
maximum: 4320
default: 900
description: Browser dimensions in pixels.
theme:
type: string
enum:
- light
- dark
description: Override the browser color scheme.
dismissCookies:
type: boolean
default: true
description: Dismiss cookie banners by accepting cookies before actions.
dismissPopups:
type: boolean
default: false
description: Dismiss other popups before actions.
actions:
type: array
minItems: 1
maxItems: 5
items:
$ref: '#/components/schemas/ScrapeAction'
description: >-
Run in order before capture. A failed action fails the request.
Bypasses caching.
waitFor:
oneOf:
- type: integer
minimum: 0
maximum: 30000
title: Milliseconds
- type: string
pattern: \S
maxLength: 2048
title: CSS selector
examples:
- 500
- '#content'
description: >-
After actions, wait this many milliseconds or until a CSS selector
is visible. Defaults to 500 ms, or 2000 ms with frames or an XML
URL. Set 0 to skip.
settleAnimations:
type: boolean
description: >-
Settle animations before capture. Defaults to true with screenshots,
otherwise false.
mainContentOnly:
type: boolean
default: false
description: Keep only main content in HTML, Markdown, images, and parsed fields.
includeSelectors:
type: array
minItems: 1
maxItems: 50
uniqueItems: true
items:
type: string
pattern: \S
maxLength: 2048
description: Keep matching content after mainContentOnly.
excludeSelectors:
type: array
minItems: 1
maxItems: 50
uniqueItems: true
items:
type: string
pattern: \S
maxLength: 2048
description: Remove matching content. Exclusions win.
includeFrames:
type: boolean
default: false
description: >-
Include iframe content in extraction. Screenshots show visible
frames regardless.
parsers:
type: object
additionalProperties: false
description: Document parsing options.
properties:
pdf:
$ref: '#/components/schemas/ScrapePdfOptions'
description: PDF text options for HTML, Markdown, and parsed fields.
ScrapeMarkdownParams:
type: object
additionalProperties: false
properties:
includeLinks:
type: boolean
default: true
includeImages:
type: boolean
default: false
inlineImages:
type: string
enum:
- placeholder
- preserve
description: >-
Base64 images use placeholders by default. Requires includeImages:
true.
allOf:
- if:
required:
- inlineImages
then:
required:
- includeImages
properties:
includeImages:
const: true
ScrapeScreenshotParams:
type: object
additionalProperties: false
properties:
area:
default: viewport
description: >-
Viewport, full page, one visible element, or a rectangle. Maximum 40
megapixels.
oneOf:
- type: string
enum:
- viewport
- fullPage
title: Page
- type: object
additionalProperties: false
properties:
selector:
type: string
pattern: \S
maxLength: 2048
description: Must match one visible element.
required:
- selector
title: Element
- type: object
additionalProperties: false
properties:
x:
type: integer
minimum: 0
maximum: 100000
'y':
type: integer
minimum: 0
maximum: 100000
width:
type: integer
minimum: 1
maximum: 7680
height:
type: integer
minimum: 1
maximum: 100000
required:
- x
- 'y'
- width
- height
title: Rectangle
description: Pixels from the document origin.
format:
type: string
enum:
- png
- jpeg
- webp
default: png
ScrapeImageParams:
type: object
additionalProperties: false
properties:
enrich:
type: array
minItems: 1
maxItems: 3
uniqueItems: true
items:
type: string
enum:
- dimensions
- classification
- file
description: >-
Add dimensions, a visual category, or a hosted file URL. Each image
has a maximum processing time of 30000 milliseconds, bounded by the
remaining request deadline.
dedupe:
type: string
enum:
- none
- visual
default: none
description: For visual duplicates, keep the largest image.
ScrapeParseParams:
type: object
additionalProperties: false
properties:
rules:
$ref: '#/components/schemas/ScrapeParseRules'
required:
- rules
ScrapeTimeout:
type: object
properties:
milliseconds:
type: integer
minimum: 1
maximum: 300000
description: 'Request deadline in milliseconds. Maximum: 300000 (5 minutes).'
behavior:
type: string
enum:
- fail
- return-partial
default: fail
description: >-
What to do at the deadline. "fail" returns 408 REQUEST_TIMEOUT
without charging credits. "return-partial" returns usable results
collected so far; if none are available, the request still fails
without charging credits. Partial results are not cached as complete
results. "return-partial" requires milliseconds of at least 5000.
required:
- milliseconds
additionalProperties: false
description: >-
Optional request deadline and behavior on timeout. For GET requests, use
timeoutOpts[milliseconds]=30000&timeoutOpts[behavior]=fail or a
JSON-encoded timeoutOpts object.
RequestTags:
type: array
items:
type: string
minLength: 1
maxLength: 50
maxItems: 20
description: >-
Optional tags for tracking usage. Up to 20 tags, each 1 to 50
characters.
example:
- production
- team-alpha
ScrapeImage:
type: object
additionalProperties: false
properties:
url:
type: string
format: uri
pattern: ^(https?://|data:image/)
description: Image URL, or a data URI for inline images.
alt:
type:
- string
- 'null'
description: Alt text, if present.
width:
type: integer
minimum: 1
height:
type: integer
minimum: 1
classification:
type: string
enum:
- photography
- illustration
- logo
- wordmark
- icon
- pattern
- graphic
- other
fileUrl:
type: string
format: uri
pattern: ^https?://
description: >-
Hosted copy when file enrichment is requested and zdr is disabled.
Valid for 24 hours from the original capture.
required:
- url
- alt
ScrapeMetadata:
type: object
additionalProperties: false
properties:
title:
type: string
description: Best title extracted from the page.
description:
type: string
description: >-
Best description extracted from standard, Open Graph, or Twitter
metadata.
language:
type: string
description: Language extracted from html lang or language meta tags.
keywords:
type: array
items:
type: string
description: Keywords extracted from the page's keywords meta tag.
canonicalUrl:
type: string
description: Resolved canonical URL, when present.
author:
type: string
description: Author metadata, when present.
siteName:
type: string
description: Site or application name from page metadata.
image:
type: string
description: >-
Primary resolved preview image from Open Graph, Twitter, or image
metadata.
favicon:
type: string
description: Resolved favicon URL, when present.
publishedTime:
type: string
description: Published timestamp/date from page metadata, when present.
modifiedTime:
type: string
description: Modified timestamp/date from page metadata, when present.
robots:
type: string
description: Robots meta directive, when present.
openGraph:
type: object
additionalProperties:
$ref: '#/components/schemas/PageMetadataValue'
description: >-
Open Graph metadata with the og: prefix removed and keys
camel-cased.
twitter:
type: object
additionalProperties:
$ref: '#/components/schemas/PageMetadataValue'
description: >-
Twitter card metadata with the twitter: prefix removed and keys
camel-cased.
alternates:
type: array
items:
$ref: '#/components/schemas/PageMetadataAlternate'
description: Resolved alternate links from link rel=alternate tags.
headings:
type: array
items:
$ref: '#/components/schemas/PageMetadataHeading'
description: >-
Page headings (h1–h6) in document order, extracted from the
unfiltered document. Capped at the first 500 headings. Omitted when
the page has none.
jsonLd:
type: array
items:
type: object
additionalProperties: true
description: JSON-LD structured data blocks parsed from the page.
additionalMeta:
type: object
additionalProperties:
$ref: '#/components/schemas/PageMetadataValue'
description: >-
Additional non-social meta tags not promoted to top-level metadata
fields.
CacheMetadata:
type: object
properties:
status:
type: string
enum:
- hit
- miss
- zdr
description: >-
Whether the response was served from cache, required fresh work, or
honored zero-data-retention cache bypass.
age_ms:
type: integer
minimum: 0
description: >-
Age of the cached data in milliseconds. Zero for miss and zdr
responses.
required:
- status
- age_ms
additionalProperties: false
description: >-
Cache outcome for this response. Composite responses are hits only when
every cache-controlled fetch contributing to the output was a hit;
age_ms is the oldest contributing hit.
RequestId:
type: string
format: uuid
description: >-
Unique id of this API call, also sent in the X-Request-Id response
header. Quote it when contacting support about a failed request.
example: 3f1c2a6e-8b4d-4c1e-9f0a-2d7b5e6c8a91
KeyMetadata:
type: object
properties:
credits_consumed:
type: integer
description: Credits used by this request.
credits_remaining:
type: integer
description: Credits remaining for your organization.
required:
- credits_consumed
- credits_remaining
description: Credit usage, included whenever a valid API key is provided.
ScrapeAction:
title: Browser action
oneOf:
- type: object
additionalProperties: false
properties:
type:
type: string
enum:
- perform
action:
type: string
minLength: 1
maxLength: 500
examples:
- Click the product details tab
required:
- type
- action
title: Instruction
- type: object
additionalProperties: false
properties:
type:
type: string
enum:
- scroll
direction:
type: string
enum:
- down
- up
- left
- right
default: down
amount:
oneOf:
- type: integer
minimum: 1
maximum: 100000
- type: string
enum:
- viewport
- max
default: viewport
selector:
type: string
pattern: \S
maxLength: 2000
description: Scroll this container. Omit to scroll the page.
maxScrolls:
type: integer
minimum: 1
maximum: 50
default: 1
required:
- type
title: Scroll
- type: object
additionalProperties: false
properties:
type:
type: string
enum:
- wait
milliseconds:
type: integer
minimum: 0
maximum: 30000
required:
- type
- milliseconds
title: Wait
- type: object
additionalProperties: false
properties:
type:
type: string
enum:
- waitFor
selector:
type: string
pattern: \S
maxLength: 2048
required:
- type
- selector
title: Wait for element
discriminator:
propertyName: type
ScrapePdfOptions:
type: object
additionalProperties: false
properties:
startPage:
type: integer
minimum: 1
description: First page to parse, starting at 1.
endPage:
type: integer
minimum: 1
description: Last page to parse. Must be at least startPage.
ocr:
type: string
enum:
- 'off'
- auto
default: 'off'
description: Read text from scanned pages.
ScrapeParseRules:
type: object
additionalProperties:
anyOf:
- type: string
minLength: 1
maxLength: 2048
- type: object
properties:
selector:
type: string
minLength: 1
maxLength: 2048
type:
type: string
enum:
- item
- list
output:
anyOf:
- type: string
enum:
- text
- html
- type: string
pattern: ^@.*$
- $ref: '#/components/schemas/ScrapeParseRules'
required:
- selector
additionalProperties: false
minProperties: 1
description: >-
Map field names to CSS selectors or rules. Missing items return null;
missing lists return [].
examples:
- title: h1
links:
selector: a
type: list
output: '@href'
PageMetadataValue:
oneOf:
- type: string
- type: array
items:
type: string
PageMetadataAlternate:
type: object
properties:
href:
type: string
description: Resolved alternate URL.
hreflang:
type: string
description: Language or locale for the alternate URL, when present.
type:
type: string
description: Alternate resource MIME type, when present.
title:
type: string
description: Alternate resource title, when present.
required:
- href
additionalProperties: false
PageMetadataHeading:
type: object
properties:
level:
type: integer
minimum: 1
maximum: 6
description: Heading level, 1–6 (from h1–h6).
text:
type: string
description: >-
Heading text with whitespace collapsed, truncated to 1000
characters.
required:
- level
- text
additionalProperties: false
headers:
RequestId:
description: >-
Unique id of this API call, sent on every response and mirrored as
request_id in JSON bodies. Quote it when contacting support.
schema:
type: string
format: uuid
RateLimitLimit:
description: Maximum requests per minute.
schema:
type: integer
minimum: 1
RateLimitRemaining:
description: Requests remaining in the current minute.
schema:
type: integer
minimum: 0
RateLimitReset:
description: Unix timestamp in seconds when the rate limit resets.
schema:
type: integer
securitySchemes:
bearerAuth:
type: http
scheme: bearer
description: >-
Bearer authentication header of the form `Bearer `. Keys have
full access by default.
````