Batches

A batch scrapes many pages as one asynchronous job. Submit up to 100,000 URLs, or a crawl from a start URL or sitemap, and you get a batch ID right away. Select Scrape Anything outputs for every page, then read the results as paginated JSON or download them as files.

What batches support

Choose a mode and one or more outputs:

Field	Values
`input.mode`	`scrape` scrapes a list of URLs (Submit a batch). `crawl` follows links from a start URL or reads a domain’s sitemap (Large crawls).
`input.data.formats`	Any combination of `markdown`, `html`, `screenshot`, `images`, `bytes`, `parse`, `product`, `highlights`, and `json`.

Batches use the same parameters and per-page result schema as Scrape Anything. See Submit a batch for extraction options, page deadlines, and pricing.

Submit a URL list

This request scrapes two pages as Markdown and HTML. The examples read your API key from CONTEXT_DEV_API_KEY; the Quickstart shows how to create a key and install an SDK.

import ContextDev from "context.dev";

const client = new ContextDev({ apiKey: process.env.CONTEXT_DEV_API_KEY });

const batch = await client.batch.submit({
  input: {
    mode: "scrape",
    data: {
      formats: { markdown: true, html: true },
      urls: [
        { url: "https://example.com/about", itemId: "about" },
        { url: "https://example.com/careers", itemId: "careers" },
      ],
    },
  },
  "Idempotency-Key": "site-import-2026-09-26",
});
console.log(batch.id, batch.status);

import os
from context.dev import ContextDev

client = ContextDev(api_key=os.environ["CONTEXT_DEV_API_KEY"])

batch = client.batch.submit(
    input={
        "mode": "scrape",
        "data": {
            "formats": {"markdown": True, "html": True},
            "urls": [
                {"url": "https://example.com/about", "item_id": "about"},
                {"url": "https://example.com/careers", "item_id": "careers"},
            ],
        },
    },
    idempotency_key="site-import-2026-09-26",
)
print(batch.id, batch.status)

require "cgi/core"
require "context_dev"

client = ContextDev::Client.new(api_key: ENV.fetch("CONTEXT_DEV_API_KEY"))

batch = client.batch.submit(
  input: {
    mode: "scrape",
    data: {
      formats: {markdown: true, html: true},
      urls: [
        {url: "https://example.com/about", item_id: "about"},
        {url: "https://example.com/careers", item_id: "careers"},
      ],
    },
  },
  idempotency_key: "site-import-2026-09-26",
)
puts batch.id, batch.status

package main

import (
	"context"
	"fmt"
	"os"

	contextdev "github.com/context-dot-dev/context-go-sdk/v2"
	"github.com/context-dot-dev/context-go-sdk/v2/option"
)

func main() {
	client := contextdev.NewClient(option.WithAPIKey(os.Getenv("CONTEXT_DEV_API_KEY")))
	body := map[string]any{"input": map[string]any{"mode": "scrape", "data": map[string]any{
		"formats": map[string]bool{"markdown": true, "html": true},
		"urls": []map[string]any{{"url": "https://example.com/about", "itemId": "about"}, {"url": "https://example.com/careers", "itemId": "careers"}},
	}}}
	var response map[string]any
	err := client.Post(context.Background(), "/batch/submit", body, &response, option.WithHeader("Idempotency-Key", "site-import-unified-v1"))
	if err != nil { panic(err) }
	fmt.Println(response["id"], response["status"])
}

<?php
require __DIR__.'/vendor/autoload.php';

use ContextDev\Client;

$client = new Client(apiKey: getenv('CONTEXT_DEV_API_KEY'));

$batch = $client->batch->submit(
    input: [
        "mode" => "scrape",
        "data" => [
            "formats" => ["markdown" => true, "html" => true],
            "urls" => [
                ["url" => "https://example.com/about", "itemID" => "about"],
                ["url" => "https://example.com/careers", "itemID" => "careers"],
            ],
        ],
    ],
    idempotencyKey: "site-import-2026-09-26",
);
echo $batch->id, " ", $batch->status, PHP_EOL;

curl https://api.context.dev/v1/batch/submit \
  -H "Authorization: Bearer $CONTEXT_DEV_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: site-import-2026-09-26" \
  -d '{
    "input": {
      "mode": "scrape",
      "data": {
        "formats": { "markdown": true, "html": true },
        "urls": [
          { "url": "https://example.com/about", "itemId": "about" },
          { "url": "https://example.com/careers", "itemId": "careers" }
        ]
      }
    }
  }'

The API returns 202 with the new batch (trimmed):

{
  "id": "batch_9f2c8a",
  "status": "queued",
  "mode": "scrape",
  "formats": { "markdown": true, "html": true },
  "input": {
    "reserved": 2,
    "reserved_is_ceiling": false,
    "submitted": 2,
    "duplicates": 0,
    "invalid": 0
  },
  "invalid_urls": [],
  "created_at": "2026-09-26T12:00:00.000Z"
}

Save id. input.submitted counts the URLs you sent. input.reserved counts the pages the batch will attempt once duplicates and invalid URLs are removed. The Idempotency-Key header makes a retried submit return this batch instead of creating a second one; see Idempotency.

Lifecycle

Status	Meaning
`queued`	Accepted and waiting to start.
`running`	Pages are being scraped.
`cancelling`	A cancel was requested. Pages already in progress finish first.
`completed`	Every accepted page was attempted. Individual pages can still have failed.
`cancelled`	Stopped by a cancel request.
`failed`	The batch itself stopped. `failure.code` says why.

completed, cancelled, and failed are final. Poll the batch or add a webhook to find out when it reaches one, then read the results. See Status and results.

Access

A restricted API key needs batches:write to submit, cancel, or delete batches and to replay their webhook deliveries. batches:read covers status, results, the batch list, and delivery history. See API keys.

Choose an endpoint

You need	Use
One page, or an output or option that batches don’t support	Scrape
Up to 500 linked pages returned in one response	Crawl
Up to 100,000 known URLs, or a crawl of more than 500 pages	A batch
A list of a site’s URLs to put in a batch	Map URLs

Submit a batch: URL lists, page options, deduplication, and idempotency.
Status and results: progress, result records, files, and how long results are kept.
Cancel, limits, and errors: cancel and delete, concurrency, and error codes.
Webhooks: signatures, retries, and replay for batch.* events.
Submit a batch reference

Get started

Web data

Company and people data

Automation

Account and billing

Production

What batches support

Submit a URL list

Lifecycle

Access

Choose an endpoint

​What batches support

​Submit a URL list

​Lifecycle

​Access

​Choose an endpoint

​Related

What batches support

Submit a URL list

Lifecycle

Access

Choose an endpoint

Related