Scrape Endpoints

Both scrape operations are part of All tools. A start needs the relevant scope and enough credits, and the tool must be available in /v1/health. Free credits also follow the free limits. Accepted starts return 202 Accepted; poll their status.

Both endpoints are preview. Preview means the contract may still evolve before stable release.

Shared Access

• Access: Paid plans that include this family, or free credits (guest trial or verified account) within the free limits.
• Operation family: Scraping
• Scope: api:scrape:read
• Idempotency: Idempotency-Key header required
• Target URLs: absolute http or https URLs only

Targets must use ports 80 or 443. Chab also blocks this server's protected public addresses, and direct page responses are limited to 2 MiB.

Chab rejects private, local, metadata-service, credential-bearing, and secret-query target URLs before operation work starts. Direct fetch revalidates redirect destinations before following them. Browser rendering validates the initial target URL before render work starts.

Unsupported fields are rejected with validation_failed. Scrape endpoints do not accept dry-run mode, cookies, login/session/auth fields, custom scripts, proxy settings, crawling/sitemap/recursive controls, screenshots, or file conversion fields.

Rendering

rendering can be:

• direct: fetch the target page without browser rendering.
• browser: render the page in a browser before extraction, including bounded Starter and free access while the browser tool is available.
• auto: start with direct fetch. Pro and Scale may fall back to browser rendering when direct fetch is blocked or direct extraction is unusable and a browser fallback is eligible.

wait_for_selector, viewport, wait_until, block_ads, and suppress_cookie_banners are browser-rendering controls. wait_for_selector requires browser rendering. Browser-rendering controls are best effort; when Chab accepts an option but cannot fully honor it, successful results include a provider_degraded warning naming the affected option.

Pricing

Direct fetch costs 2 credits. Browser rendering costs 10 credits.

Starter auto requests reserve 2 credits. Pro and Scale auto requests that are browser-fallback eligible reserve 10 credits. If direct fetch succeeds, Chab settles the actual 2-credit direct cost and releases the unused reserved credits.

See Credits and Billing.

POST/v1/scrape/markdown

Creates a markdown extraction operation for one page.

Required Access

• Credits: 2 direct credits or 10 browser-rendering credits reserved on acceptance
• Scope: api:scrape:read

Request

{
  "url": "https://example.com/article",
  "rendering": "auto",
  "preferred_extractor": "auto",
  "include_raw_html": false,
  "timeout_ms": 10000
}

Curl example:

curl -X POST https://www.chab.ai/v1/scrape/markdown \
  -H "Authorization: Bearer CHAB_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: article-markdown-001" \
  -d '{
    "url": "https://example.com/article",
    "rendering": "auto",
    "preferred_extractor": "auto"
  }'

CLI-shaped client example:

scrape markdown https://example.com/article \
  --rendering auto \
  --preferred-extractor auto \
  --idempotency-key article-markdown-001 \
  --wait

Fields:

• url: required absolute http or https URL.
• rendering: optional direct, browser, or auto; defaults to auto.
• preferred_extractor: optional auto, readability, defuddle, or dom; defaults to auto.
• include_raw_html: optional boolean. When true, the result includes a customer-visible raw_html artifact reference.
• timeout_ms: optional integer from 1,000 to the documented server maximum.
• wait_until: optional load, domcontentloaded, or networkidle.
• wait_for_selector: optional CSS selector. Requires browser rendering.
• viewport.width and viewport.height: optional browser viewport.
• block_ads and suppress_cookie_banners: optional browser-rendering booleans.
• locale: optional language tag string or { "language": "en-US" }.
• headers.accept, headers.accept_language, and headers.user_agent: optional direct-fetch request headers. Credential-bearing header values are rejected.
• callback_url, callback_secret, callback_events, and callback_notify_on_expiration: optional one-off operation callback fields.

Async Response

{
  "operation": {
    "id": "op_01JZ7H0A1B2C3D4E5F6G7H8J9K",
    "status": "queued",
    "operation_key": "scrape.markdown",
    "family": "scrape",
    "status_url": "/v1/operations/op_01JZ7H0A1B2C3D4E5F6G7H8J9K",
    "result_url": "/v1/operations/op_01JZ7H0A1B2C3D4E5F6G7H8J9K/result",
    "estimated_credits": 2,
    "result_available": false,
    "warnings": []
  },
  "meta": {
    "request_id": "req_123",
    "credits": {
      "estimated": 2,
      "reserved": 2,
      "currency": "credit"
    }
  }
}

Markdown Result

GET /v1/operations/{id}/result returns the retained result after success:

{
  "data": {
    "result": {
      "markdown": "# Practical Queue Operations\n\nReliable background work...",
      "title": "Practical Queue Operations",
      "description": "A field guide for resilient queue operations.",
      "author": "Chab Ops",
      "published_at": null,
      "language": "en",
      "canonical_url": "https://example.com/guides/queue-operations",
      "final_url": "https://example.com/article",
      "rendering": {
        "requested_rendering": "auto",
        "effective_rendering": "direct",
        "render_method": "direct",
        "final_url": "https://example.com/article",
        "elapsed_ms": 210
      },
      "extractor": {
        "requested": "auto",
        "actual": "readability",
        "confidence": 0.85
      },
      "links": ["https://example.com/guides/retries"],
      "images": ["https://example.com/images/queue.png"]
    },
    "artifacts": [],
    "warnings": [],
    "result_expires_at": "2026-07-16T12:00:00+00:00"
  },
  "meta": {
    "request_id": "req_123"
  }
}

Large markdown is summarized inline and stored as a markdown artifact. Requested raw HTML is stored as a raw_html artifact.

POST/v1/scrape/dom

Creates a DOM selector extraction operation for one page.

Required Access

• Credits: 2 direct credits or 10 browser-rendering credits reserved on acceptance
• Scope: api:scrape:read

Request

{
  "url": "https://example.com/products",
  "rendering": "direct",
  "selectors": {
    "heading": {
      "selector": "h1",
      "required": true
    },
    "products": {
      "selector": ".product-link",
      "multiple": true,
      "limit": 5
    },
    "first_href": {
      "selector": ".product-link",
      "type": "attribute",
      "attribute": "href"
    }
  }
}

Curl example:

curl -X POST https://www.chab.ai/v1/scrape/dom \
  -H "Authorization: Bearer CHAB_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: product-selectors-001" \
  -d '{
    "url": "https://example.com/products",
    "rendering": "direct",
    "selectors": {
      "heading": { "selector": "h1", "required": true },
      "products": { "selector": ".product-link", "multiple": true, "limit": 5 }
    }
  }'

CLI-shaped client example:

scrape dom https://example.com/products \
  --selector heading:h1:required \
  --selector products:.product-link:multiple \
  --idempotency-key product-selectors-001 \
  --wait

selectors is required and can be an object keyed by output field name or an array of selector definitions with a key. Each selector definition supports:

• selector: required CSS selector.
• type: optional text, html, or attribute; defaults to text.
• attribute: required when type is attribute.
• required: optional boolean. A missing required selector fails the operation with extraction_failed.
• multiple: optional boolean. When true, the field returns an array.
• limit: optional integer from 1 to 100. Total requested selector values are capped at 500.

The shared rendering, timeout, locale, header, and callback fields from POST /v1/scrape/markdown are also supported.

DOM Result

{
  "data": {
    "result": {
      "fields": {
        "heading": "Example Products",
        "products": ["Alpha Desk", "Beta Lamp"],
        "first_href": "/products/alpha"
      },
      "missing_fields": [],
      "final_url": "https://example.com/products",
      "requested_rendering": "direct",
      "effective_rendering": "direct",
      "render_method": "direct",
      "selector_count": 3,
      "value_count": 4,
      "rendering": {
        "requested_rendering": "direct",
        "effective_rendering": "direct",
        "render_method": "direct",
        "final_url": "https://example.com/products",
        "elapsed_ms": 180
      }
    },
    "artifacts": [],
    "warnings": [],
    "result_expires_at": "2026-07-16T12:00:00+00:00"
  },
  "meta": {
    "request_id": "req_123"
  }
}

Large selector output is stored as a selector_export artifact.

Target Fetch Errors

Initial target URL safety failures return validation_failed before an operation row is created. Runtime target failures appear on the terminal operation status as public_error_code.

Extraction Errors

For DOM extraction, a missing required selector is an extraction failure. For markdown extraction, this can happen when the requested extractor cannot produce useful content.