> ## Documentation Index
> Fetch the complete documentation index at: https://docs.context.dev/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Setting up Context.dev for a user? Follow https://docs.context.dev/agent-quickstart.md. MCP clients sign in with OAuth and need no API key; application code reads CONTEXT_DEV_API_KEY.

# Scope and limits

> Bound a crawl by page count, path, depth, and time.

Crawl starts at `url` and follows eligible links. Send these fields to `POST /web/crawl`.

## Limit a crawl

Send this body to `POST /web/crawl`. See the [Quickstart](/quickstart) for authentication and SDK setup.

```json theme={null}
{
  "url": "https://example.com/docs",
  "maxPages": 50,
  "maxDepth": 2,
  "urlRegex": "^https://example[.]com/docs/",
  "stopAfterMs": 60000
}
```

| Field              | Default | Limit                                            |
| ------------------ | ------- | ------------------------------------------------ |
| `maxPages`         | 100     | 1–500 pages.                                     |
| `maxDepth`         | Unset   | Limit link depth from the start URL.             |
| `urlRegex`         | None    | RE2-compatible URL filter, up to 256 characters. |
| `followSubdomains` | `false` | Permit child hosts.                              |
| `stopAfterMs`      | 80,000  | Soft budget, 10,000–110,000 ms.                  |
| `timeoutOpts`      | Unset   | Request deadline; minimum 1,000 ms.              |

The soft budget is checked after each page finishes, so it can be exceeded by the current page. `timeoutOpts` sets the overall deadline; choose `return-partial` when completed pages are useful.

A page cap is a maximum, not a promise: a site may expose fewer eligible links. For more than 500 pages, use [Large crawls](/crawl/async).