> ## Documentation Index > Fetch the complete documentation index at: https://docs.context.dev/llms.txt > Use this file to discover all available pages before exploring further. # Scrape Markdown > Convert any page into clean Markdown optimized for agents, RAG, search, and analysis. 1 Credit ## OpenAPI ````yaml GET /web/scrape/markdown openapi: 3.0.0 info: title: Context API description: API for retrieving context data from any website version: 1.0.0 servers: - url: https://api.context.dev/v1 security: [] tags: - name: Monitors description: >- Monitor pages, sitemaps, and extracted website data for exact or semantic changes. The change.detected webhook payload is documented by the MonitorsChangeDetectedWebhookPayload schema. paths: /web/scrape/markdown: get: tags: - Web Scraping summary: Scrape Markdown description: Scrapes the given URL into LLM usable Markdown. parameters: - name: url in: query required: true schema: type: string format: uri description: >- Full URL to scrape into LLM usable Markdown (must include http:// or https:// protocol) - name: includeLinks in: query required: false schema: type: boolean default: true description: Preserve hyperlinks in Markdown output - name: includeImages in: query required: false schema: type: boolean default: false description: Include image references in Markdown output - name: shortenBase64Images in: query required: false schema: type: boolean default: true description: Shorten base64-encoded image data in the Markdown output - name: useMainContentOnly in: query required: false schema: type: boolean default: false description: >- Extract only the main content of the page, excluding headers, footers, sidebars, and navigation - name: pdf in: query required: false style: form explode: false schema: type: object properties: shouldParse: type: boolean default: true description: >- When true, PDF URLs are fetched and parsed. When false, PDF URLs are skipped and a 400 WEBSITE_ACCESS_ERROR is returned. start: type: integer minimum: 1 description: >- First 1-based PDF page to parse. When omitted, parsing starts at the first page. end: type: integer minimum: 1 description: >- Last 1-based PDF page to parse. When omitted, parsing ends at the last page. Must be greater than or equal to start when both are provided. additionalProperties: false default: shouldParse: true description: >- PDF parsing controls. Use start/end to limit text extraction and OCR to an inclusive 1-based page range. - name: includeFrames in: query required: false schema: type: boolean default: false description: When true, the contents of iframes are rendered to Markdown. - name: includeSelectors in: query required: false style: form explode: true schema: type: array maxItems: 50 items: type: string maxLength: 2048 description: >- CSS selectors. When provided, only matching HTML subtrees (and their descendants) are kept before conversion to Markdown. When omitted, the entire document is kept. Examples: "article.main", "#content", "[role=main]". - name: excludeSelectors in: query required: false style: form explode: true schema: type: array maxItems: 50 items: type: string maxLength: 2048 description: >- CSS selectors to remove before conversion to Markdown. Applied after includeSelectors. Exclusion takes precedence: an element matching both is removed. Examples: "nav", "footer", ".ad-banner", "[aria-hidden=true]". - name: maxAgeMs in: query required: false schema: type: integer minimum: 0 maximum: 2592000000 default: 86400000 description: >- Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh. - name: waitForMs in: query required: false schema: type: integer minimum: 0 maximum: 30000 description: >- Optional browser wait time in milliseconds after initial page load before converting the page to Markdown. Min: 0. Max: 30000 (30 seconds). - name: settleAnimations in: query required: false schema: type: boolean default: false description: >- When true, waits briefly for CSS and transition animations to settle before converting to Markdown. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages. - $ref: '#/components/parameters/ScrapeHeaders' - $ref: '#/components/parameters/BrowserCountry' - $ref: '#/components/parameters/TimeoutMS' responses: '200': description: Successful response content: application/json: schema: type: object properties: success: type: boolean enum: - true description: Indicates success markdown: type: string description: Page content converted to GitHub Flavored Markdown url: type: string description: The URL that was scraped metadata: $ref: '#/components/schemas/PageMetadata' key_metadata: $ref: '#/components/schemas/KeyMetadata' required: - success - markdown - url - metadata '400': description: Bad request - Invalid URL or failed to scrape content: application/json: schema: type: object properties: message: type: string description: Error message describing the issue error_code: type: string enum: - INPUT_VALIDATION_ERROR - WEBSITE_ACCESS_ERROR description: Error code indicating the type of error key_metadata: $ref: '#/components/schemas/KeyMetadata' required: - message - error_code '401': description: Unauthorized - Invalid or missing API key content: application/json: schema: type: object properties: message: type: string description: Error message error_code: type: string enum: - UNAUTHORIZED description: Error code indicating unauthorized access key_metadata: $ref: '#/components/schemas/KeyMetadata' '403': description: Forbidden - Insufficient permissions or usage limit exceeded content: application/json: schema: type: object properties: message: type: string description: Error message error_code: type: string enum: - FORBIDDEN - USAGE_EXCEEDED - DISABLED - INSUFFICIENT_PERMISSIONS description: Error code indicating forbidden access key_metadata: $ref: '#/components/schemas/KeyMetadata' '404': description: Target page returned a 404 content: application/json: schema: type: object properties: message: type: string description: Error message describing the issue error_code: type: string enum: - NOT_FOUND description: Error code indicating the target page was not found key_metadata: $ref: '#/components/schemas/KeyMetadata' required: - message - error_code '408': description: Request timeout content: application/json: schema: type: object properties: message: type: string description: Timeout error message error_code: type: string enum: - REQUEST_TIMEOUT description: Error code indicating request timeout key_metadata: $ref: '#/components/schemas/KeyMetadata' '415': description: >- Unsupported content type - the URL resolved to a content type that is not supported (e.g. an image, presentation, media, or archive). Supported types are HTML, XML, PDF, DOCX, DOC, XLSX, XLS, PPTX, PPT, and CSV. content: application/json: schema: type: object properties: message: type: string description: Error message describing the unsupported content type error_code: type: string enum: - UNSUPPORTED_CONTENT description: Error code indicating an unsupported content type key_metadata: $ref: '#/components/schemas/KeyMetadata' required: - message - error_code '429': $ref: '#/components/responses/RateLimited' '500': description: Internal server error content: application/json: schema: type: object properties: message: type: string description: Error message error_code: type: string enum: - INTERNAL_ERROR description: Error code indicating internal server error key_metadata: $ref: '#/components/schemas/KeyMetadata' security: - bearerAuth: [] x-codeSamples: - lang: JavaScript source: >- import ContextDev from 'context.dev'; const client = new ContextDev({ apiKey: process.env['CONTEXT_DEV_API_KEY'], // This is the default and can be omitted }); const response = await client.web.webScrapeMd({ url: 'https://example.com' }); console.log(response.markdown); - lang: Python source: |- import os from context.dev import ContextDev client = ContextDev( api_key=os.environ.get("CONTEXT_DEV_API_KEY"), # This is the default and can be omitted ) response = client.web.web_scrape_md( url="https://example.com", ) print(response.markdown) - lang: Go source: "package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/context-dot-dev/context-go-sdk\"\n\t\"github.com/context-dot-dev/context-go-sdk/option\"\n)\n\nfunc main() {\n\tclient := contextdev.NewClient(\n\t\toption.WithAPIKey(\"My API Key\"),\n\t)\n\tresponse, err := client.Web.WebScrapeMd(context.TODO(), contextdev.WebWebScrapeMdParams{\n\t\tURL: \"https://example.com\",\n\t})\n\tif err != nil {\n\t\tpanic(err.Error())\n\t}\n\tfmt.Printf(\"%+v\\n\", response.Markdown)\n}\n" - lang: Ruby source: |- require "context_dev" context_dev = ContextDev::Client.new(api_key: "My API Key") response = context_dev.web.web_scrape_md(url: "https://example.com") puts(response) - lang: PHP source: >- web->webScrapeMd( url: 'https://example.com', country: 'de', excludeSelectors: ['string'], headers: ['foo' => 'J!'], includeFrames: true, includeImages: true, includeLinks: true, includeSelectors: ['string'], maxAgeMs: 0, pdf: ['end' => 1, 'shouldParse' => true, 'start' => 1], settleAnimations: true, shortenBase64Images: true, timeoutMs: 1000, useMainContentOnly: true, waitForMs: 0, ); var_dump($response); } catch (APIException $e) { echo $e->getMessage(); } - lang: CLI source: |- context-dev web web-scrape-md \ --api-key 'My API Key' \ --url https://example.com components: parameters: ScrapeHeaders: name: headers in: query required: false style: form explode: false schema: $ref: '#/components/schemas/HeaderMap' description: >- Optional outbound HTTP headers forwarded only to the target URL, sent as deep-object query params such as headers[X-Custom]=value. When provided, caching is bypassed: the result is neither read from nor written to cache. BrowserCountry: name: country in: query required: false schema: $ref: '#/components/schemas/BrowserCountryCode' description: >- Two-letter ISO 3166-1 alpha-2 country code for the website request location. When provided, Context.dev fetches the target page from that country. TimeoutMS: name: timeoutMS in: query required: false schema: $ref: '#/components/schemas/TimeoutMS' description: >- Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes). schemas: PageMetadata: type: object description: Metadata extracted from the scraped page HTML. properties: sourceUrl: type: string description: Original URL requested by the caller. finalUrl: type: string description: >- Final URL scraped after redirects or scraper fallback, when known. Falls back to sourceUrl when unavailable. title: type: string description: Best title extracted from the page. description: type: string description: >- Best description extracted from standard, Open Graph, or Twitter metadata. language: type: string description: Language extracted from html lang or language meta tags. keywords: type: array description: Keywords extracted from the page's keywords meta tag. items: type: string canonicalUrl: type: string description: Resolved canonical URL, when present. author: type: string description: Author metadata, when present. siteName: type: string description: Site or application name from page metadata. image: type: string description: >- Primary resolved preview image from Open Graph, Twitter, or image metadata. favicon: type: string description: Resolved favicon URL, when present. publishedTime: type: string description: Published timestamp/date from page metadata, when present. modifiedTime: type: string description: Modified timestamp/date from page metadata, when present. robots: type: string description: Robots meta directive, when present. openGraph: type: object description: >- Open Graph metadata with the og: prefix removed and keys camel-cased. additionalProperties: $ref: '#/components/schemas/PageMetadataValue' twitter: type: object description: >- Twitter card metadata with the twitter: prefix removed and keys camel-cased. additionalProperties: $ref: '#/components/schemas/PageMetadataValue' alternates: type: array description: Resolved alternate links from link rel=alternate tags. items: $ref: '#/components/schemas/PageMetadataAlternate' jsonLd: type: array description: JSON-LD structured data blocks parsed from the page. items: type: object additionalProperties: true additionalMeta: type: object description: >- Additional non-social meta tags not promoted to top-level metadata fields. additionalProperties: $ref: '#/components/schemas/PageMetadataValue' required: - sourceUrl - finalUrl KeyMetadata: type: object description: >- Metadata about the API key used for the request. Included in every response whenever a valid API key is provided, even when the response status is not 200. properties: credits_consumed: type: integer description: The number of credits consumed by this request. credits_remaining: type: integer description: >- The number of credits remaining for your organization after this request. required: - credits_consumed - credits_remaining HeaderMap: type: object additionalProperties: type: string maxLength: 8192 pattern: ^[^\r\n]*$ BrowserCountryCode: type: string enum: - ad - ae - af - ag - ai - al - am - ao - ar - at - au - aw - az - ba - bb - bd - be - bf - bg - bh - bi - bj - bm - bn - bo - bq - br - bs - bw - by - bz - ca - cd - cf - cg - ch - ci - cl - cm - cn - co - cr - cv - cw - cy - cz - de - dj - dk - dm - do - dz - ec - ee - eg - es - et - fi - fj - fr - ga - gb - gd - ge - gf - gg - gh - gm - gn - gp - gq - gr - gt - gu - gw - gy - hk - hn - hr - ht - hu - id - ie - il - im - in - iq - ir - is - it - je - jm - jo - jp - ke - kg - kh - kn - kr - kw - ky - kz - la - lb - lc - lk - lr - ls - lt - lu - lv - ly - ma - mc - md - me - mf - mg - mk - ml - mm - mn - mo - mq - mr - mt - mu - mv - mw - mx - my - mz - na - nc - ne - ng - ni - nl - 'no' - np - nz - om - pa - pe - pf - pg - ph - pk - pl - pr - ps - pt - py - qa - re - ro - rs - ru - rw - sa - sc - sd - se - sg - si - sk - sl - sm - sn - so - sr - ss - st - sv - sx - sy - sz - tc - td - tg - th - tj - tl - tm - tn - tr - tt - tw - tz - ua - ug - us - uy - uz - vc - ve - vg - vi - vn - ye - yt - za - zm - zw example: de description: >- Two-letter ISO 3166-1 alpha-2 country code identifying a supported Context.dev residential proxy exit location. Must be one of Context.dev's supported countries. When provided, Context.dev fetches the target page from that country. TimeoutMS: type: integer minimum: 1000 maximum: 300000 description: >- Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes). PageMetadataValue: oneOf: - type: string - type: array items: type: string PageMetadataAlternate: type: object properties: href: type: string description: Resolved alternate URL. hreflang: type: string description: Language or locale for the alternate URL, when present. type: type: string description: Alternate resource MIME type, when present. title: type: string description: Alternate resource title, when present. required: - href additionalProperties: false responses: RateLimited: description: Rate limit exceeded headers: Retry-After: description: Seconds until the per-minute rate limit window resets schema: type: integer minimum: 1 maximum: 60 content: application/json: schema: type: object properties: message: type: string description: Error message error_code: type: string enum: - RATE_LIMITED description: Error code indicating the rate limit was exceeded key_metadata: $ref: '#/components/schemas/KeyMetadata' required: - message - error_code securitySchemes: bearerAuth: type: http scheme: bearer ````