> ## Documentation Index
> Fetch the complete documentation index at: https://docs.context.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Scrape Markdown

> Convert any page into clean Markdown optimized for agents, RAG, search, and analysis.

<Badge color="blue">1 Credit</Badge>


## OpenAPI

````yaml GET /web/scrape/markdown
openapi: 3.1.0
info:
  title: Context API
  description: API for retrieving context data from any website
  version: 1.0.0
servers:
  - url: https://api.context.dev/v1
security: []
tags:
  - name: Monitors
    description: >-
      Monitor pages, sitemaps, and extracted website data for exact or semantic
      changes. Webhook payloads are documented by the
      MonitorsChangeDetectedWebhookPayload and
      MonitorsRunCompletedWebhookPayload schemas.
paths:
  /web/scrape/markdown:
    get:
      tags:
        - Web Scraping
      summary: Scrape Markdown
      description: >-
        Scrapes the given URL into LLM usable Markdown. Inspect key_metadata on
        JSON responses from a recognized API key; use error_code to distinguish
        stable failure categories.


        ### Billing & errors


        | HTTP status | Billed? | Meaning |

        | --- | --- | --- |

        | 200 | Yes — 1 credit | Successful scrape, including a zero-length
        result when includeSelectors matched nothing |

        | 400 | No | Invalid input, skipped PDF, or the page could not be
        scraped |

        | 401 / 403 | No | Invalid/disabled key, insufficient permissions, or
        credits exhausted; inspect error_code |

        | 404 | No | Target page returned or fingerprinted as not found |

        | 408 | No | Request timed out |

        | 415 | No | Unsupported content type |

        | 429 | No | Per-minute rate limit exceeded; honor Retry-After |

        | 500 | No | Internal error |
      parameters:
        - name: url
          in: query
          required: true
          schema:
            type: string
            format: uri
          description: >-
            Full URL to scrape into LLM usable Markdown (must include http:// or
            https:// protocol)
        - name: includeLinks
          in: query
          required: false
          schema:
            type: boolean
            default: true
          description: Preserve hyperlinks in Markdown output
        - name: includeImages
          in: query
          required: false
          schema:
            type: boolean
            default: false
          description: Include image references in Markdown output
        - name: shortenBase64Images
          in: query
          required: false
          schema:
            type: boolean
            default: true
          description: Shorten base64-encoded image data in the Markdown output
        - name: useMainContentOnly
          in: query
          required: false
          schema:
            type: boolean
            default: false
          description: >-
            Extract only the main content of the page, excluding headers,
            footers, sidebars, and navigation
        - name: pdf
          in: query
          required: false
          style: form
          explode: false
          schema:
            type: object
            properties:
              shouldParse:
                type: boolean
                default: true
                description: >-
                  When true, PDF URLs are fetched and parsed. When false, PDF
                  URLs are skipped and a 400 WEBSITE_ACCESS_ERROR is returned.
              ocr:
                type: boolean
                default: false
                description: >-
                  When true, detect and OCR images embedded in the selected PDF
                  pages, inserting recognized text at each image's position in
                  page reading order while preserving the PDF text layer. This
                  is separate from automatic scanned-PDF OCR fallback.
              start:
                type: integer
                minimum: 1
                description: >-
                  First 1-based PDF page to parse. When omitted, parsing starts
                  at the first page.
              end:
                type: integer
                minimum: 1
                description: >-
                  Last 1-based PDF page to parse. When omitted, parsing ends at
                  the last page. Must be greater than or equal to start when
                  both are provided.
            additionalProperties: false
            default:
              shouldParse: true
              ocr: false
          description: >-
            PDF parsing controls. Use start/end to limit text extraction and
            embedded-image detection/OCR to an inclusive 1-based page range.
        - name: includeFrames
          in: query
          required: false
          schema:
            type: boolean
            default: false
          description: When true, the contents of iframes are rendered to Markdown.
        - name: includeSelectors
          in: query
          required: false
          style: form
          explode: true
          schema:
            type: array
            maxItems: 50
            items:
              type: string
              maxLength: 2048
          description: >-
            CSS selectors. When provided, only matching HTML subtrees (and their
            descendants) are kept before conversion to Markdown. When omitted,
            the entire document is kept. Examples: "article.main", "#content",
            "[role=main]".
        - name: excludeSelectors
          in: query
          required: false
          style: form
          explode: true
          schema:
            type: array
            maxItems: 50
            items:
              type: string
              maxLength: 2048
          description: >-
            CSS selectors to remove before conversion to Markdown. Applied after
            includeSelectors. Exclusion takes precedence: an element matching
            both is removed. Examples: "nav", "footer", ".ad-banner",
            "[aria-hidden=true]".
        - name: maxAgeMs
          in: query
          required: false
          schema:
            type: integer
            minimum: 0
            maximum: 2592000000
            default: 86400000
          description: >-
            Return a cached result if a prior scrape for the same parameters
            exists and is younger than this many milliseconds. Defaults to 1 day
            (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0
            to always scrape fresh.
        - name: waitForMs
          in: query
          required: false
          schema:
            type: integer
            minimum: 0
            maximum: 30000
          description: >-
            Optional browser wait time in milliseconds after initial page load
            before converting the page to Markdown. Min: 0. Max: 30000 (30
            seconds). 
        - name: settleAnimations
          in: query
          required: false
          schema:
            type: boolean
            default: false
          description: >-
            When true, waits briefly for CSS and transition animations to settle
            before converting to Markdown. Defaults to false. This adds a bit of
            latency in exchange for more stable output on animated pages.
        - $ref: '#/components/parameters/ScrapeHeaders'
        - $ref: '#/components/parameters/BrowserCountry'
        - $ref: '#/components/parameters/TimeoutMS'
        - $ref: '#/components/parameters/RequestTags'
      responses:
        '200':
          description: Successful response
          headers:
            X-RateLimit-Limit:
              $ref: '#/components/headers/RateLimitLimit'
            X-RateLimit-Remaining:
              $ref: '#/components/headers/RateLimitRemaining'
            X-RateLimit-Reset:
              $ref: '#/components/headers/RateLimitReset'
          content:
            application/json:
              schema:
                type: object
                properties:
                  success:
                    type: boolean
                    enum:
                      - true
                    description: Indicates success
                  markdown:
                    type: string
                    description: Page content converted to GitHub Flavored Markdown
                  contentLength:
                    type: integer
                    minimum: 0
                    description: >-
                      UTF-8 byte length of the returned Markdown. Use 0 to
                      identify an empty result and compare small values against
                      your workload's minimum useful-content threshold.
                  url:
                    type: string
                    description: The URL that was scraped
                  metadata:
                    $ref: '#/components/schemas/PageMetadata'
                  key_metadata:
                    $ref: '#/components/schemas/KeyMetadata'
                required:
                  - success
                  - markdown
                  - contentLength
                  - url
                  - metadata
        '400':
          description: Bad request - Invalid URL or failed to scrape
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string
                    description: Error message describing the issue
                  error_code:
                    type: string
                    enum:
                      - INPUT_VALIDATION_ERROR
                      - WEBSITE_ACCESS_ERROR
                    description: Error code indicating the type of error
                  key_metadata:
                    $ref: '#/components/schemas/KeyMetadata'
                required:
                  - message
                  - error_code
          headers:
            X-RateLimit-Limit:
              $ref: '#/components/headers/RateLimitLimit'
            X-RateLimit-Remaining:
              $ref: '#/components/headers/RateLimitRemaining'
            X-RateLimit-Reset:
              $ref: '#/components/headers/RateLimitReset'
        '401':
          description: Unauthorized - Invalid or missing API key
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string
                    description: Error message
                  error_code:
                    type: string
                    enum:
                      - UNAUTHORIZED
                    description: Error code indicating unauthorized access
                  key_metadata:
                    $ref: '#/components/schemas/KeyMetadata'
          headers:
            X-RateLimit-Limit:
              $ref: '#/components/headers/RateLimitLimit'
            X-RateLimit-Remaining:
              $ref: '#/components/headers/RateLimitRemaining'
            X-RateLimit-Reset:
              $ref: '#/components/headers/RateLimitReset'
        '403':
          description: Forbidden - Insufficient permissions or usage limit exceeded
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string
                    description: Error message
                  error_code:
                    type: string
                    enum:
                      - FORBIDDEN
                      - USAGE_EXCEEDED
                      - DISABLED
                      - INSUFFICIENT_PERMISSIONS
                    description: Error code indicating forbidden access
                  key_metadata:
                    $ref: '#/components/schemas/KeyMetadata'
          headers:
            X-RateLimit-Limit:
              $ref: '#/components/headers/RateLimitLimit'
            X-RateLimit-Remaining:
              $ref: '#/components/headers/RateLimitRemaining'
            X-RateLimit-Reset:
              $ref: '#/components/headers/RateLimitReset'
        '404':
          description: Target page returned a 404
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string
                    description: Error message describing the issue
                  error_code:
                    type: string
                    enum:
                      - NOT_FOUND
                    description: Error code indicating the target page was not found
                  key_metadata:
                    $ref: '#/components/schemas/KeyMetadata'
                required:
                  - message
                  - error_code
          headers:
            X-RateLimit-Limit:
              $ref: '#/components/headers/RateLimitLimit'
            X-RateLimit-Remaining:
              $ref: '#/components/headers/RateLimitRemaining'
            X-RateLimit-Reset:
              $ref: '#/components/headers/RateLimitReset'
        '408':
          description: Request timeout
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string
                    description: Timeout error message
                  error_code:
                    type: string
                    enum:
                      - REQUEST_TIMEOUT
                    description: Error code indicating request timeout
                  key_metadata:
                    $ref: '#/components/schemas/KeyMetadata'
          headers:
            X-RateLimit-Limit:
              $ref: '#/components/headers/RateLimitLimit'
            X-RateLimit-Remaining:
              $ref: '#/components/headers/RateLimitRemaining'
            X-RateLimit-Reset:
              $ref: '#/components/headers/RateLimitReset'
        '415':
          description: >-
            Unsupported content type - the URL resolved to a content type that
            is not supported (e.g. an image, presentation, media, or archive).
            Supported types are HTML, XML, PDF, DOCX, DOC, XLSX, XLS, PPTX, PPT,
            and CSV.
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string
                    description: Error message describing the unsupported content type
                  error_code:
                    type: string
                    enum:
                      - UNSUPPORTED_CONTENT
                    description: Error code indicating an unsupported content type
                  key_metadata:
                    $ref: '#/components/schemas/KeyMetadata'
                required:
                  - message
                  - error_code
          headers:
            X-RateLimit-Limit:
              $ref: '#/components/headers/RateLimitLimit'
            X-RateLimit-Remaining:
              $ref: '#/components/headers/RateLimitRemaining'
            X-RateLimit-Reset:
              $ref: '#/components/headers/RateLimitReset'
        '429':
          $ref: '#/components/responses/RateLimited'
        '500':
          description: Internal server error
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string
                    description: Error message
                  error_code:
                    type: string
                    enum:
                      - INTERNAL_ERROR
                    description: Error code indicating internal server error
                  key_metadata:
                    $ref: '#/components/schemas/KeyMetadata'
          headers:
            X-RateLimit-Limit:
              $ref: '#/components/headers/RateLimitLimit'
            X-RateLimit-Remaining:
              $ref: '#/components/headers/RateLimitRemaining'
            X-RateLimit-Reset:
              $ref: '#/components/headers/RateLimitReset'
      security:
        - bearerAuth: []
      x-codeSamples:
        - lang: JavaScript
          source: >-
            import ContextDev from 'context.dev';


            const client = new ContextDev({
              apiKey: process.env['CONTEXT_DEV_API_KEY'], // This is the default and can be omitted
            });


            const response = await client.web.webScrapeMd({ url:
            'https://example.com' });


            console.log(response.contentLength);
        - lang: Python
          source: |-
            import os
            from context.dev import ContextDev

            client = ContextDev(
                api_key=os.environ.get("CONTEXT_DEV_API_KEY"),  # This is the default and can be omitted
            )
            response = client.web.web_scrape_md(
                url="https://example.com",
            )
            print(response.content_length)
        - lang: Go
          source: "package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/context-dot-dev/context-go-sdk\"\n\t\"github.com/context-dot-dev/context-go-sdk/option\"\n)\n\nfunc main() {\n\tclient := contextdev.NewClient(\n\t\toption.WithAPIKey(\"My API Key\"),\n\t)\n\tresponse, err := client.Web.WebScrapeMd(context.TODO(), contextdev.WebWebScrapeMdParams{\n\t\tURL: \"https://example.com\",\n\t})\n\tif err != nil {\n\t\tpanic(err.Error())\n\t}\n\tfmt.Printf(\"%+v\\n\", response.ContentLength)\n}\n"
        - lang: Ruby
          source: |-
            require "context_dev"

            context_dev = ContextDev::Client.new(api_key: "My API Key")

            response = context_dev.web.web_scrape_md(url: "https://example.com")

            puts(response)
        - lang: PHP
          source: >-
            <?php


            require_once dirname(__DIR__) . '/vendor/autoload.php';


            use ContextDev\Client;

            use ContextDev\Core\Exceptions\APIException;


            $client = new Client(apiKey: getenv('CONTEXT_DEV_API_KEY') ?: 'My
            API Key');


            try {
              $response = $client->web->webScrapeMd(
                url: 'https://example.com',
                country: 'de',
                excludeSelectors: ['string'],
                headers: ['foo' => 'J!'],
                includeFrames: true,
                includeImages: true,
                includeLinks: true,
                includeSelectors: ['string'],
                maxAgeMs: 0,
                pdf: ['end' => 1, 'ocr' => true, 'shouldParse' => true, 'start' => 1],
                settleAnimations: true,
                shortenBase64Images: true,
                tags: ['production', 'team-alpha'],
                timeoutMs: 1000,
                useMainContentOnly: true,
                waitForMs: 0,
              );

              var_dump($response);
            } catch (APIException $e) {
              echo $e->getMessage();
            }
        - lang: CLI
          source: |-
            context-dev web web-scrape-md \
              --api-key 'My API Key' \
              --url https://example.com
components:
  parameters:
    ScrapeHeaders:
      name: headers
      in: query
      required: false
      style: form
      explode: false
      schema:
        $ref: '#/components/schemas/HeaderMap'
      description: >-
        Optional outbound HTTP headers forwarded only to the target URL, sent as
        deep-object query params such as headers[X-Custom]=value. When provided,
        caching is bypassed: the result is neither read from nor written to
        cache.
    BrowserCountry:
      name: country
      in: query
      required: false
      schema:
        $ref: '#/components/schemas/BrowserCountryCode'
      description: >-
        Two-letter ISO 3166-1 alpha-2 country code for the website request
        location. When provided, Context.dev fetches the target page from that
        country.
    TimeoutMS:
      name: timeoutMS
      in: query
      required: false
      schema:
        $ref: '#/components/schemas/TimeoutMS'
      description: >-
        Optional timeout in milliseconds for the request. If the request takes
        longer than this value, it will be aborted with a 408 status code.
        Maximum allowed value is 300000ms (5 minutes).
    RequestTags:
      name: tags
      in: query
      required: false
      style: form
      explode: false
      schema:
        $ref: '#/components/schemas/RequestTags'
      description: >-
        Optional comma-separated caller-defined tags for tracking this request.
        Tags are recorded on the request's usage log and can be used to filter
        usage on the dashboard usage page. Up to 20 tags, each 1-50 characters.
      example: production,team-alpha
  headers:
    RateLimitLimit:
      description: >-
        Maximum requests allowed in the current fixed one-minute window.
        Returned when the authenticated API key has a per-minute rate limit.
      schema:
        type: integer
        minimum: 1
    RateLimitRemaining:
      description: >-
        Requests remaining in the current fixed one-minute window. Returned when
        the authenticated API key has a per-minute rate limit.
      schema:
        type: integer
        minimum: 0
    RateLimitReset:
      description: >-
        Unix timestamp in seconds when the current rate-limit window resets.
        Returned when the authenticated API key has a per-minute rate limit.
      schema:
        type: integer
  schemas:
    PageMetadata:
      type: object
      description: Metadata extracted from the scraped page HTML.
      properties:
        sourceUrl:
          type: string
          description: Original URL requested by the caller.
        finalUrl:
          type: string
          description: >-
            Final URL scraped after redirects or scraper fallback, when known.
            Falls back to sourceUrl when unavailable.
        title:
          type: string
          description: Best title extracted from the page.
        description:
          type: string
          description: >-
            Best description extracted from standard, Open Graph, or Twitter
            metadata.
        language:
          type: string
          description: Language extracted from html lang or language meta tags.
        keywords:
          type: array
          description: Keywords extracted from the page's keywords meta tag.
          items:
            type: string
        canonicalUrl:
          type: string
          description: Resolved canonical URL, when present.
        author:
          type: string
          description: Author metadata, when present.
        siteName:
          type: string
          description: Site or application name from page metadata.
        image:
          type: string
          description: >-
            Primary resolved preview image from Open Graph, Twitter, or image
            metadata.
        favicon:
          type: string
          description: Resolved favicon URL, when present.
        publishedTime:
          type: string
          description: Published timestamp/date from page metadata, when present.
        modifiedTime:
          type: string
          description: Modified timestamp/date from page metadata, when present.
        robots:
          type: string
          description: Robots meta directive, when present.
        openGraph:
          type: object
          description: >-
            Open Graph metadata with the og: prefix removed and keys
            camel-cased.
          additionalProperties:
            $ref: '#/components/schemas/PageMetadataValue'
        twitter:
          type: object
          description: >-
            Twitter card metadata with the twitter: prefix removed and keys
            camel-cased.
          additionalProperties:
            $ref: '#/components/schemas/PageMetadataValue'
        alternates:
          type: array
          description: Resolved alternate links from link rel=alternate tags.
          items:
            $ref: '#/components/schemas/PageMetadataAlternate'
        jsonLd:
          type: array
          description: JSON-LD structured data blocks parsed from the page.
          items:
            type: object
            additionalProperties: true
        additionalMeta:
          type: object
          description: >-
            Additional non-social meta tags not promoted to top-level metadata
            fields.
          additionalProperties:
            $ref: '#/components/schemas/PageMetadataValue'
      required:
        - sourceUrl
        - finalUrl
    KeyMetadata:
      type: object
      description: >-
        Metadata about the API key used for the request. Included in every
        response whenever a valid API key is provided, even when the response
        status is not 200.
      properties:
        credits_consumed:
          type: integer
          description: The number of credits consumed by this request.
        credits_remaining:
          type: integer
          description: >-
            The number of credits remaining for your organization after this
            request.
      required:
        - credits_consumed
        - credits_remaining
    HeaderMap:
      type: object
      additionalProperties:
        type: string
        maxLength: 8192
        pattern: ^[^\r\n]*$
    BrowserCountryCode:
      type: string
      enum:
        - ad
        - ae
        - af
        - ag
        - ai
        - al
        - am
        - ao
        - ar
        - at
        - au
        - aw
        - az
        - ba
        - bb
        - bd
        - be
        - bf
        - bg
        - bh
        - bi
        - bj
        - bm
        - bn
        - bo
        - bq
        - br
        - bs
        - bw
        - by
        - bz
        - ca
        - cd
        - cf
        - cg
        - ch
        - ci
        - cl
        - cm
        - cn
        - co
        - cr
        - cv
        - cw
        - cy
        - cz
        - de
        - dj
        - dk
        - dm
        - do
        - dz
        - ec
        - ee
        - eg
        - es
        - et
        - fi
        - fj
        - fr
        - ga
        - gb
        - gd
        - ge
        - gf
        - gg
        - gh
        - gm
        - gn
        - gp
        - gq
        - gr
        - gt
        - gu
        - gw
        - gy
        - hk
        - hn
        - hr
        - ht
        - hu
        - id
        - ie
        - il
        - im
        - in
        - iq
        - ir
        - is
        - it
        - je
        - jm
        - jo
        - jp
        - ke
        - kg
        - kh
        - kn
        - kr
        - kw
        - ky
        - kz
        - la
        - lb
        - lc
        - lk
        - lr
        - ls
        - lt
        - lu
        - lv
        - ly
        - ma
        - mc
        - md
        - me
        - mf
        - mg
        - mk
        - ml
        - mm
        - mn
        - mo
        - mq
        - mr
        - mt
        - mu
        - mv
        - mw
        - mx
        - my
        - mz
        - na
        - nc
        - ne
        - ng
        - ni
        - nl
        - 'no'
        - np
        - nz
        - om
        - pa
        - pe
        - pf
        - pg
        - ph
        - pk
        - pl
        - pr
        - ps
        - pt
        - py
        - qa
        - re
        - ro
        - rs
        - ru
        - rw
        - sa
        - sc
        - sd
        - se
        - sg
        - si
        - sk
        - sl
        - sm
        - sn
        - so
        - sr
        - ss
        - st
        - sv
        - sx
        - sy
        - sz
        - tc
        - td
        - tg
        - th
        - tj
        - tl
        - tm
        - tn
        - tr
        - tt
        - tw
        - tz
        - ua
        - ug
        - us
        - uy
        - uz
        - vc
        - ve
        - vg
        - vi
        - vn
        - ye
        - yt
        - za
        - zm
        - zw
      example: de
      description: >-
        Two-letter ISO 3166-1 alpha-2 country code identifying a supported
        Context.dev residential proxy exit location. Must be one of
        Context.dev's supported countries. When provided, Context.dev fetches
        the target page from that country.
    TimeoutMS:
      type: integer
      minimum: 1000
      maximum: 300000
      description: >-
        Optional timeout in milliseconds for the request. If the request takes
        longer than this value, it will be aborted with a 408 status code.
        Maximum allowed value is 300000ms (5 minutes).
    RequestTags:
      type: array
      items:
        type: string
        minLength: 1
        maxLength: 50
      maxItems: 20
      description: >-
        Optional caller-defined tags for tracking this request. Tags are
        recorded on the request's usage log and can be used to filter usage on
        the dashboard usage page. Up to 20 tags, each 1-50 characters.
      example:
        - production
        - team-alpha
    PageMetadataValue:
      oneOf:
        - type: string
        - type: array
          items:
            type: string
    PageMetadataAlternate:
      type: object
      properties:
        href:
          type: string
          description: Resolved alternate URL.
        hreflang:
          type: string
          description: Language or locale for the alternate URL, when present.
        type:
          type: string
          description: Alternate resource MIME type, when present.
        title:
          type: string
          description: Alternate resource title, when present.
      required:
        - href
      additionalProperties: false
  responses:
    RateLimited:
      description: Rate limit exceeded
      headers:
        X-RateLimit-Limit:
          $ref: '#/components/headers/RateLimitLimit'
        X-RateLimit-Remaining:
          $ref: '#/components/headers/RateLimitRemaining'
        X-RateLimit-Reset:
          $ref: '#/components/headers/RateLimitReset'
        Retry-After:
          description: Seconds until the per-minute rate limit window resets
          schema:
            type: integer
            minimum: 1
            maximum: 60
      content:
        application/json:
          schema:
            type: object
            properties:
              message:
                type: string
                description: Error message
              error_code:
                type: string
                enum:
                  - RATE_LIMITED
                description: Error code indicating the rate limit was exceeded
              key_metadata:
                $ref: '#/components/schemas/KeyMetadata'
            required:
              - message
              - error_code
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer

````