> ## Documentation Index
> Fetch the complete documentation index at: https://docs.context.dev/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Setting up Context.dev for a user? Follow https://docs.context.dev/agent-quickstart.md. MCP clients sign in with OAuth and need no API key; application code reads CONTEXT_DEV_API_KEY.

# Page content

> Choose how each crawled page becomes Markdown.

Crawl options live at the top level of the request. Their names differ from Scrape: use `useMainContentOnly` and `waitForMs`, not Scrape’s `sharedParams` fields. Unknown Crawl fields can be ignored, so check names carefully.

## Keep article content

Send this body to `POST /web/crawl`. See the [Quickstart](/quickstart) for authentication and SDK setup.

```json theme={null}
{
  "url": "https://example.com/blog",
  "maxPages": 10,
  "useMainContentOnly": true,
  "excludeSelectors": [
    "nav",
    "footer"
  ],
  "includeLinks": true,
  "includeImages": false
}
```

| Field                                  | Default | Effect                                            |
| -------------------------------------- | ------- | ------------------------------------------------- |
| `useMainContentOnly`                   | `false` | Remove detected page chrome.                      |
| `includeSelectors`, `excludeSelectors` | None    | Keep or remove matching elements; exclusions win. |
| `includeLinks`                         | `true`  | Keep link destinations.                           |
| `includeImages`                        | `false` | Include image references.                         |
| `shortenBase64Images`                  | `true`  | Replace inline base64 payloads with placeholders. |
| `includeFrames`                        | `false` | Read iframe content.                              |
| `waitForMs`                            | 3,500   | Wait before extraction.                           |
| `settleAnimations`                     | `false` | Wait for CSS animations.                          |
| `maxAgeMs`                             | One day | Accept cached page content; `0` fetches fresh.    |

Use `pdf: {"shouldParse":true,"ocr":true,"start":1,"end":5}` for selected PDF pages. Page numbers are inclusive and start at 1. OCR recovers scanned pages while preserving existing text.

`country` selects the fetch location, `zdr` applies [zero data retention](/optimization/zero-data-retention), and `tags` label the request in [usage logs](/optimization/usage-and-logs).