Skip to main content
A batch scrapes many pages as one asynchronous job. Submit up to 100,000 URLs, or a crawl from a start URL or sitemap, and you get a batch ID right away. Select Scrape Anything outputs for every page, then read the results as paginated JSON or download them as files.

What batches support

Choose a mode and one or more outputs: Batches use the same parameters and per-page result schema as Scrape Anything. See Submit a batch for extraction options, page deadlines, and pricing. Add policies to apply access policies to every page.

Submit a URL list

This request scrapes two pages as Markdown and HTML. The examples read your API key from CONTEXT_DEV_API_KEY; the Quickstart shows how to create a key and install an SDK.
The API returns 202 with the new batch (trimmed):
Save id. input.submitted counts the URLs you sent. input.reserved counts the pages the batch will attempt once duplicates and invalid URLs are removed, and credits.reserved is what was held for them. invalid_urls lists up to 100 rejected URLs, each with a reason such as Must be a public http:// or https:// URL. A submission with a webhook also returns webhook_secret, which only this response includes. The Idempotency-Key header makes a retried submit return this batch instead of creating a second one; see Idempotency.

Lifecycle

completed, cancelled, and failed are final. Poll the batch or add a webhook to find out when it reaches one, then read the results. See Status and results.

Access

A restricted API key needs batches:write to submit, cancel, or delete batches and to replay their webhook deliveries. batches:read covers status, results, the batch list, and delivery history. See API keys.

Choose an endpoint