Batches

A batch scrapes many pages as one asynchronous job. Submit up to 25,000 URLs, or a crawl from a start URL or sitemap, and you get a batch ID right away. When the batch finishes, read each page’s Markdown or HTML as paginated JSON or download it as files.

What batches support

Every batch has one mode and one output format:

Field	Values
`input.mode`	`scrape` scrapes a list of URLs (Submit a batch). `crawl` follows links from a start URL or reads a domain’s sitemap (Large crawls).
`input.data.format`	`markdown` or `html`.

Batches don’t return screenshots, images, bytes, parsed fields, highlights, JSON, or product data. They also don’t accept browser actions, custom headers, zdr, or timeoutOpts. For any of those, call Scrape once per URL.

Submit a URL list

This request scrapes two pages as Markdown. The examples read your API key from CONTEXT_DEV_API_KEY; the Quickstart shows how to create a key and install an SDK.

import ContextDev from "context.dev";

const client = new ContextDev({ apiKey: process.env.CONTEXT_DEV_API_KEY });

const batch = await client.batch.submit({
  input: {
    mode: "scrape",
    data: {
      format: "markdown",
      urls: [
        { url: "https://example.com/about", itemId: "about" },
        { url: "https://example.com/careers", itemId: "careers" },
      ],
    },
  },
  "Idempotency-Key": "site-import-2026-09-26",
});
console.log(batch.id, batch.status);

import os
from context.dev import ContextDev

client = ContextDev(api_key=os.environ["CONTEXT_DEV_API_KEY"])

batch = client.batch.submit(
    input={
        "mode": "scrape",
        "data": {
            "format": "markdown",
            "urls": [
                {"url": "https://example.com/about", "item_id": "about"},
                {"url": "https://example.com/careers", "item_id": "careers"},
            ],
        },
    },
    idempotency_key="site-import-2026-09-26",
)
print(batch.id, batch.status)

require "cgi/core"
require "context_dev"

client = ContextDev::Client.new(api_key: ENV.fetch("CONTEXT_DEV_API_KEY"))

batch = client.batch.submit(
  input: {
    mode: "scrape",
    data: {
      format: "markdown",
      urls: [
        {url: "https://example.com/about", item_id: "about"},
        {url: "https://example.com/careers", item_id: "careers"},
      ],
    },
  },
  idempotency_key: "site-import-2026-09-26",
)
puts batch.id, batch.status

package main

import (
	"context"
	"fmt"
	"os"

	contextdev "github.com/context-dot-dev/context-go-sdk/v2"
	"github.com/context-dot-dev/context-go-sdk/v2/option"
)

func main() {
	client := contextdev.NewClient(option.WithAPIKey(os.Getenv("CONTEXT_DEV_API_KEY")))

	batch, err := client.Batch.Submit(context.Background(), contextdev.BatchSubmitParams{
		Input: contextdev.BatchSubmitParamsInputUnion{
			OfScrape: &contextdev.BatchSubmitParamsInputScrape{
				Mode: "scrape",
				Data: contextdev.BatchSubmitParamsInputScrapeDataUnion{
					OfMarkdown: &contextdev.BatchSubmitParamsInputScrapeDataMarkdown{
						Format: "markdown",
						URLs: []contextdev.BatchSubmitParamsInputScrapeDataMarkdownURL{
							{URL: "https://example.com/about", ItemID: contextdev.String("about")},
							{URL: "https://example.com/careers", ItemID: contextdev.String("careers")},
						},
					},
				},
			},
		},
		IdempotencyKey: contextdev.String("site-import-2026-09-26"),
	})
	if err != nil {
		panic(err)
	}
	fmt.Println(batch.ID, batch.Status)
}

<?php
require __DIR__.'/vendor/autoload.php';

use ContextDev\Client;

$client = new Client(apiKey: getenv('CONTEXT_DEV_API_KEY'));

$batch = $client->batch->submit(
    input: [
        "mode" => "scrape",
        "data" => [
            "format" => "markdown",
            "urls" => [
                ["url" => "https://example.com/about", "itemID" => "about"],
                ["url" => "https://example.com/careers", "itemID" => "careers"],
            ],
        ],
    ],
    idempotencyKey: "site-import-2026-09-26",
);
echo $batch->id, " ", $batch->status, PHP_EOL;

curl https://api.context.dev/v1/batch/submit \
  -H "Authorization: Bearer $CONTEXT_DEV_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: site-import-2026-09-26" \
  -d '{
    "input": {
      "mode": "scrape",
      "data": {
        "format": "markdown",
        "urls": [
          { "url": "https://example.com/about", "itemId": "about" },
          { "url": "https://example.com/careers", "itemId": "careers" }
        ]
      }
    }
  }'

The API returns 202 with the new batch (trimmed):

{
  "id": "batch_9f2c8a",
  "status": "queued",
  "mode": "scrape",
  "format": "markdown",
  "input": {
    "reserved": 2,
    "reserved_is_ceiling": false,
    "submitted": 2,
    "duplicates": 0,
    "invalid": 0
  },
  "invalid_urls": [],
  "created_at": "2026-09-26T12:00:00.000Z"
}

Save id. input.submitted counts the URLs you sent. input.reserved counts the pages the batch will attempt once duplicates and invalid URLs are removed. The Idempotency-Key header makes a retried submit return this batch instead of creating a second one; see Idempotency.

Lifecycle

Status	Meaning
`queued`	Accepted and waiting to start.
`running`	Pages are being scraped.
`cancelling`	A cancel was requested. Pages already in progress finish first.
`completed`	Every accepted page was attempted. Individual pages can still have failed.
`cancelled`	Stopped by a cancel request.
`failed`	The batch itself stopped. `failure.code` says why.

completed, cancelled, and failed are final. Poll the batch or add a webhook to find out when it reaches one, then read the results. See Status and results.

Access

A restricted API key needs batches:write to submit, cancel, or delete batches and to replay their webhook deliveries. batches:read covers status, results, the batch list, and delivery history. See API keys.

Choose an endpoint

You need	Use
One page, or an output or option that batches don’t support	Scrape
Up to 500 linked pages returned in one response	Crawl
Up to 25,000 known URLs, or a crawl of more than 500 pages	A batch
A list of a site’s URLs to put in a batch	Map URLs

Submit a batch: URL lists, page options, deduplication, and idempotency.
Status and results: progress, result records, files, and how long results are kept.
Cancel, limits, and errors: cancel and delete, concurrency, and error codes.
Webhooks: signatures, retries, and replay for batch.* events.
Submit a batch reference

Get started

Web data

Company and people data

Automation

Account and billing

Production

What batches support

Submit a URL list

Lifecycle

Access

Choose an endpoint

​What batches support

​Submit a URL list

​Lifecycle

​Access

​Choose an endpoint

​Related

What batches support

Submit a URL list

Lifecycle

Access

Choose an endpoint

Related