> ## Documentation Index
> Fetch the complete documentation index at: https://docs.context.dev/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Setting up Context.dev for a user? Follow https://docs.context.dev/agent-quickstart.md. MCP clients sign in with OAuth and need no API key; application code reads CONTEXT_DEV_API_KEY.

# Submit a batch

> Submit URL lists with extraction options and safe retry keys.

Submit 1–25,000 URL records to `POST /batch/submit`. Use the [Batches overview](/batches/overview#submit-a-url-list) for runnable SDK examples, or [Large crawls](/crawl/async) for discovery from a site.

## URL records and format

```json theme={null}
{
  "input": {
    "mode": "scrape",
    "data": {
      "format": "markdown",
      "urls": [
        {
          "url": "https://example.com/about",
          "itemId": "about",
          "meta": {
            "section": "company"
          }
        }
      ],
      "options": {
        "includeHTML": true,
        "useMainContentOnly": true
      }
    }
  },
  "tags": [
    "site-import"
  ]
}
```

`itemId` and `meta` are returned with each page. Duplicate URLs are deduplicated; rejected URLs appear in `invalid_urls` (up to 100 entries). Keep the request body within 8 MB and use public HTTP(S) URLs.

Choose one format: `markdown` or `html`. `includeHTML` adds HTML alongside Markdown. Other Scrape outputs, browser actions, custom headers, ZDR, and request deadlines are not supported by batches.

## Per-page options

| `input.data.options` field                             | Default or behavior                                                                                                            |
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `useMainContentOnly`                                   | `false`.                                                                                                                       |
| `includeHTML`                                          | `false`; Markdown format only.                                                                                                 |
| `includeSelectors`, `excludeSelectors`                 | Up to 50 selectors each, 2,048 characters per selector; force fresh fetches.                                                   |
| `maxAgeMs`                                             | Three days (`259200000` ms), maximum 365 days (`31536000000` ms); `0` fetches fresh. Applies to both scrape and crawl batches. |
| `waitForMs`                                            | 0–15,000 ms; `0` uses 500 ms.                                                                                                  |
| `settleAnimations`                                     | `false`.                                                                                                                       |
| `country`                                              | Supported fetch country.                                                                                                       |
| `pdf`                                                  | `shouldParse`, `ocr`, and inclusive `start`/`end` pages.                                                                       |
| `includeLinks`, `includeImages`, `shortenBase64Images` | Markdown formatting options.                                                                                                   |

Use up to 20 request tags. Configure `webhook: {"url":"https://app.example.com/hooks/context","retry":{}}` for [signed completion events](/webhooks); do not send `webhookUrl` at the same time.

## Idempotency

Send an `Idempotency-Key` header, up to 200 characters, unique within your organization. Retrying the same key and body returns the original batch; a different body returns 409. Keep the key and payload together when retrying an uncertain submission.

## Recover selected pages

Read [result records](/batches/results#read-results), select failed URLs, and submit them as a new batch with a new idempotency key. Use `maxAgeMs: 0` for a fresh price list, or remove it when recent cached pages are acceptable. Check [limits and errors](/batches/limits-and-errors) before increasing concurrency.