API Documentation

Machine-readable API capabilities. AI orchestrators use this to discover features.

Response includes:

• All capabilities with inline examples
• Authentication methods
• Rate limits and pricing
• Idempotency configuration

API index listing all endpoint categories.

Check your usage quota before starting crawls.

{
  "pages_remaining": 9500,
  "crawls_remaining": 47,
  "can_start_crawl": true,
  "plan": "pro"
}

Basic health check of all services.

{
  "dashboard": "ok",
  "database": "ok",
  "crawler": "ok",
  "redis": "ok"
}

Detailed health with latency and capacity info. Useful for orchestrator decisions.

{
  "status": "healthy",
  "version": "10.1.0",
  "dependencies": {
    "database": {"status": "healthy", "latency_ms": 5},
    "redis": {"status": "healthy", "latency_ms": 2},
    "crawler": {"status": "healthy", "latency_ms": 15}
  },
  "capacity": {
    "active_crawls": 3,
    "max_concurrent_crawls": 10,
    "available_slots": 7
  }
}

Login to get JWT token.

{
  "email": "[email protected]",
  "password": "your-password"
}

List your API keys.

Create a new scoped API key.

{
  "name": "CI/CD Pipeline",
  "scopes": ["read", "write"],
  "expires_in_days": 90
}

Available Scopes

• read - View jobs, pages, reports
• write - Start crawls, create webhooks
• admin - Manage users, delete resources

Revoke an API key.

Delete multiple jobs at once (max 100).

{
  "job_ids": ["abc123", "def456", "ghi789"]
}

Export multiple jobs at once (max 20).

{
  "job_ids": ["abc123", "def456"],
  "format": "json",
  "scope": "summary"  // or "pages"
}

List your webhooks.

Create a webhook to receive event notifications.

{
  "name": "Slack Notification",
  "url": "https://hooks.slack.com/...",
  "events": ["crawl.completed", "crawl.failed"],
  "secret": "optional-hmac-secret"
}

Available Events

Send a test event to verify your webhook endpoint.

Delete a webhook.

Get webhook details

Update webhook configuration

Get delivery history

WebSocket endpoint for real-time progress streaming.

// Connect
const ws = new WebSocket('ws://host/api/crawl/jobs/abc123/stream');

// Receive updates
ws.onmessage = (e) => {
  const msg = JSON.parse(e.data);
  // msg.type: "progress" | "complete" | "error"
  // msg.data: { job_id, status, percent_complete, ... }
};

Start a new crawl job. All config options default to minimal footprint settings.

Request Body

{
  "url": "https://example.com",
  "workspace_id": 1,
  
  // Basic Settings
  "max_pages": 500,                    // Max internal pages (external links don't count)
  "max_external_links": 0,            // Max external links to check (0 = unlimited)
  "max_depth": 3,
  "delay_ms": 100,
  "timeout_seconds": 30,
  "concurrent_requests": 5,
  "max_redirects": 5,
  
  // Scope Settings
  "crawl_subdomains": false,
  "include_external": true,           // Track external link URLs
  "crawl_external": false,            // Check external links (HEAD request)
  "crawl_outside_start_folder": true,
  "respect_robots_txt": true,
  "store_content": false,             // Store HTML (required for word counts)
  
  // Analysis Settings (all default to false for minimal footprint)
  "seo_analysis": false,
  "performance_metrics": false,
  "detect_duplicates": false,
  "track_redirects": false,
  "extract_hreflang": false,
  "extract_structured_data": false,
  
  // Resource Tracking (all default to false)
  // store_* = Track URLs only, crawl_* = HEAD request to check size
  "store_images": false,   "crawl_images": false,
  "store_css": false,      "crawl_css": false,
  "store_js": false,       "crawl_js": false,
  "store_fonts": false,    "crawl_fonts": false,
  "store_media": false,    "crawl_media": false,
  "store_other": false,    "crawl_other": false,
  
  // URL Filtering
  "include_patterns": [],
  "exclude_patterns": [],
  "remove_parameters": ["utm_source", "utm_medium", "fbclid"],
  
  // Authentication (optional)
  "credential_id": null,
  "formauth_id": null,
  
  // JavaScript Rendering (optional)
  "js_rendering": false,
  "js_browser_type": "chromium",       // chromium, firefox, webkit
  "js_device_preset": "desktop_1080p",
  "js_wait_until": "networkidle",
  
  // Screenshots (optional, requires js_rendering)
  "capture_screenshots": false,
  "screenshot_full_page": false,
  "screenshot_format": "png"
}

Device Presets

Available values for js_device_preset:

Response

{
  "job_id": "abc12345",
  "status": "started"
}

List all crawl jobs with pagination and filtering.

Query Parameters

Get detailed information about a specific job.

Delete a crawl job and all associated data.

Update crawl job metadata

Start recrawl of existing job

Cancel a running crawl

Get crawl version history

Get real-time crawl progress

Get crawl overview dashboard data

Get crawl execution logs

Check status of job deletions

Get all pages from a crawl job.

Query Parameters

Get detailed page data including content, headers, and links.

Get crawl statistics and metrics.

Get full HTML content of page

Get page screenshot

Get page thumbnail

List all screenshots for crawl

Delete page screenshot

Delete all screenshots for crawl

Get audit statistics

Get auditable pages

Get audit details for page

Submit audit decision

Get all audit decisions

Get audit task for page

Update audit task

Export audit results

Get task for sitemap node

Update sitemap node task

Bulk update node tasks

Get improvement suggestions

Generate new suggestions

Apply suggestion

Dismiss suggestion

Create shared crawl link

Access shared crawl data

Get pages for content editing

Get content for editing

Save content version

Get content version history

Restore content version

Bulk update content

Get content brief

Update content brief

Get page brief override

Update page brief

Delete page brief

Register new user account.

Logout and invalidate session.

Verify email address via token.

Request password reset email.

Reset password with token.

Redirect to OAuth provider (google, github).

Google OAuth callback.

GitHub OAuth callback.

Update user profile.

Change current password.

Create new API key.

Get notification preferences.

Update notification preferences.

Get MFA enrollment status.

Start MFA setup (generates QR code).

Complete MFA setup with TOTP code.

Disable MFA.

Verify MFA code during login.

Send MFA verification code via email.

Regenerate backup codes.

Set organization MFA enforcement policy.

List trusted devices.

Remove trusted device.

Delete user account.

List available SSO providers.

List SSO connections for organization.

Create new SSO connection.

Get SSO connection details.

Update SSO connection.

Delete SSO connection.

Test SSO configuration.

Get SAML service provider metadata.

Get governance policy for workspace.

Create or update governance policy.

List queued governance changes.

Approve a queued change.

Reject a queued change.

Mark change as manually fixed.

Rollback an applied change.

Bulk approve entire batch.

Bulk reject entire batch.

Bulk mark batch as fixed.

Rollback entire batch.

Query governance activity log.

Get activity for specific batch.

Ungated healthcheck for shared AEO infrastructure. Does not depend on any tab feature flag.

{
  "ok": true,
  "feature_flag": null
}

Mint a verification token for a domain on the authenticated user's workspace. Idempotent — repeated calls for the same (workspace_id, domain) return the same token.

Query Parameters

• workspace_id (integer, required) — workspace the caller is a member of.

{
  "success": true,
  "data": {
    "domain": "example.com",
    "token": "<url-safe-base64, 64 chars>",
    "instructions": "Add a TXT record at _iato-verify.example.com with the token as the value, then call GET .../verify/status."
  }
}

Check whether the DNS TXT record at _iato-verify.<domain> matches the workspace's minted token. Updates last_checked_at; sets verified_at on first successful match.

Query Parameters

• workspace_id (integer, required) — workspace the caller is a member of.

{
  "success": true,
  "data": {
    "domain": "example.com",
    "verified": false,
    "verified_at": null,
    "last_checked_at": "2026-05-22T10:42:18Z"
  }
}

tab is one of: overview, access, content, grounding, visibility.

Query Parameters

• workspace_id (integer, required) — the workspace whose feature-flag state is being probed.

{
  "ok": true,
  "feature_flag": true
}

Per-face per-layer AEO sub-scores for a single crawl. Returns both engine faces (Google AI Search readiness + Other AI Engines readiness) with a clamped 0–100 total plus a six-key by_layer issue-count breakdown (Access / Discovery / Capability / Content / Tokens / Visibility). Reads live seo_issues rows for the crawl and aggregates per-face per-layer. workspace_id is derived server-side from crawl_jobs.workspace_id — the endpoint accepts no workspace_id query parameter.

Each face also carries measurable_layers — the subset of layers that have at least one IssueType mapped to this face server-side. Layers absent from this list are "not measured for this face" (rendered as gray on the sub-score grid); layers present in the list with a zero by_layer count are "awaiting analysis" until producers run (rendered as amber). Through M2 the Google face's measurable_layers is ["Content", "Tokens"] and the Other face's is all six layers, reflecting the locked engine-face partition.

{
  "success": true,
  "data": {
    "crawl_id": 11237,
    "workspace_id": 17,
    "google": {
      "total": 100,
      "by_layer": {
        "Access": 0, "Discovery": 0, "Capability": 0,
        "Content": 0, "Tokens": 0, "Visibility": 0
      },
      "measurable_layers": ["Content", "Tokens"]
    },
    "other": {
      "total": 100,
      "by_layer": {
        "Access": 0, "Discovery": 0, "Capability": 0,
        "Content": 0, "Tokens": 0, "Visibility": 0
      },
      "measurable_layers": [
        "Access", "Discovery", "Capability",
        "Content", "Tokens", "Visibility"
      ]
    }
  }
}

Failure responses

• 401 unauthorized — API key missing or invalid (auth middleware).
• 404 crawl_not_found — crawl_id (UUID) does not match any crawl_jobs row.
• 403 not_workspace_member — caller is not a member of the crawl's owning workspace.
• 403 feature_disabled — aeo.overview flag is off for the workspace.

Up to five prioritized AEO recommendations for the crawl, ordered by priority (high → medium → low) then effort (low → medium → high). Reads ai_suggestions filtered to suggestion_type LIKE 'aeo_%' and status = 'pending'. Through M2 with producers stubbed, the array is empty — the Top-5 panel renders the locked empty-state copy ("No recommendations yet — AEO issues will surface here as they're detected."). Real recommendations land as M3+ producers populate seo_issues with aeo_* IssueType rows and the suggestion generator emits matching ai_suggestions rows.

Each recommendation carries applies_to engine-face attribution (["google"], ["other"], or ["google", "other"]) and layer (one of the six AEO layers, or null for unknown IssueTypes). Both fields are derived server-side at read time from the static _LAYER_MAPPING partition — not persisted in the database. auto_fixable defaults false for all aeo_* rows in M2; per-type tuning is M4+ work.

{
  "success": true,
  "data": {
    "crawl_id": 11237,
    "workspace_id": 17,
    "recommendations": [
      {
        "id": 4821,
        "suggestion_type": "aeo_perplexity_blocked",
        "issue_type": "aeo_perplexity_blocked",
        "title": "Perplexity bot is blocked",
        "description": "Improves visibility to non-Google AI engines that surface citations from Perplexity's index.",
        "priority": "high",
        "effort": "medium",
        "applies_to": ["other"],
        "layer": "Access",
        "auto_fixable": false,
        "affected_count": 1,
        "created_at": "2026-05-26T09:47:25Z"
      }
    ]
  }
}

Failure responses

• 401 unauthorized — API key missing or invalid (auth middleware).
• 404 crawl_not_found — crawl_id (UUID) does not match any crawl_jobs row.
• 403 not_workspace_member — caller is not a member of the crawl's owning workspace.
• 403 feature_disabled — aeo.overview flag is off for the workspace.

Per-face time-series of completed crawls' total AEO scores within a backward time window in the same workspace as the URL crawl. Powers the score trend chart on the Tab 1 Overview surface. Both face arrays (data.google and data.other) are aligned by index (one entry per scored crawl in the window, repeated across both faces) and ordered ASC by x (the score's computed_at timestamp). Through M2 every scored crawl returns 100 for both faces, so the chart renders a flat line at 100; M3+ producers populate real variation naturally.

Query Parameters

• days (integer, optional, default 30, range 1–365) — backward time window for the trend in days. Values outside the 1–365 range return 400 invalid_window.

Dual crawl_id semantics

The response carries two crawl_id fields at different tree levels with different meanings — both intentional. data.crawl_id (top level) echoes the resolved numeric crawl_jobs.id from the URL UUID and identifies the crawl the operator is currently viewing. Each point's crawl_id inside data.google[] / data.other[] is the crawl_jobs.id of the crawl that produced THAT specific score — typically a different value, since the trend includes prior crawls of the same workspace.

{
  "success": true,
  "data": {
    "crawl_id": 11237,
    "workspace_id": 17,
    "window_days": 30,
    "google": [
      { "x": "2026-05-08T12:34:56Z", "y": 100, "crawl_id": 11189 },
      { "x": "2026-05-16T09:15:22Z", "y": 100, "crawl_id": 11214 },
      { "x": "2026-05-26T10:02:11Z", "y": 100, "crawl_id": 11237 }
    ],
    "other": [
      { "x": "2026-05-08T12:34:56Z", "y": 100, "crawl_id": 11189 },
      { "x": "2026-05-16T09:15:22Z", "y": 100, "crawl_id": 11214 },
      { "x": "2026-05-26T10:02:11Z", "y": 100, "crawl_id": 11237 }
    ]
  }
}

Failure responses

• 400 invalid_window — days is less than 1 or greater than 365. Fires before workspace-ownership lookup.
• 401 unauthorized — API key missing or invalid (auth middleware).
• 404 crawl_not_found — crawl_id (UUID) does not match any crawl_jobs row.
• 403 not_workspace_member — caller is not a member of the crawl's owning workspace.
• 403 feature_disabled — aeo.overview flag is off for the workspace.

Multi-agent access matrix for a single crawl. Rows are pages crawled, columns are AI agents probed, cells are status badges keyed by the page_agent_access.status ENUM. The matrix surfaces how each of nine canonical AI crawler user-agents accessed your pages during the multi-UA probe pass.

Query Parameters

• agents (string, optional) — comma-separated UA preset names to include as columns. Default: all nine presets in canonical source order (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, anthropic-ai, Applebot-Extended, cohere-ai). Unknown names are dropped silently; an all-unknown filter falls back to the full nine.
• limit (integer, optional, default 50, range 1–200) — page-axis page size. Pagination is offset-based on the page axis; each pages[] entry is one page, with its cells inline (cells are not paginated separately).
• offset (integer, optional, default 0, range ≥0) — page-axis offset.

Status badges

Cell status mirrors the page_agent_access.status ENUM plus the synthesized no_data:

• ok — agent fetched the page successfully (2xx).
• blocked — access refused at HTTP layer (401 / 451).
• partial — partial-content response (206).
• http_403 / http_429 — explicit HTTP-status badges for 403 (often AI-bot-block) and 429 (rate-limited).
• timeout — no response within budget.
• error — connection / parse / other failure.
• no_data — synthesized; this agent did not probe this page (no page_agent_access row exists).

Token-only UAs

Three of the nine presets — Google-Extended, anthropic-ai, Applebot-Extended — are robots.txt-only tokens AI vendors publish so site operators can gate training-data ingestion separately from search/answer crawlers. No real HTTP crawler ships these exact UAs. The matrix probes them with synthetic compatible; <token>/1.0-style UA strings so server logs can branch on the token name, but the HTTP results do not reflect what real AI vendors see when they ingest the page. The response identifies these three in the data.token_only_agents array; the dashboard UI renders them in italics with a "robots.txt only" subtitle to make the caveat visible. Real per-token robots.txt semantics ship in a later release.

{
  "success": true,
  "data": {
    "crawl_id": 11262,
    "workspace_id": 17,
    "agents": [
      "GPTBot", "OAI-SearchBot", "ClaudeBot", "PerplexityBot",
      "Google-Extended", "Bytespider", "anthropic-ai",
      "Applebot-Extended", "cohere-ai"
    ],
    "token_only_agents": ["Applebot-Extended", "Google-Extended", "anthropic-ai"],
    "pages": [
      {
        "page_id": 220605,
        "url": "https://example.com/",
        "cells": {
          "GPTBot":            { "status": "ok",
                                 "http_status": 200,
                                 "response_time_ms": 27,
                                 "blocked_by": null,
                                 "fetched_at": "2026-05-29T00:17:25Z" },
          "OAI-SearchBot":     { "status": "no_data" },
          "ClaudeBot":         { "status": "no_data" },
          "PerplexityBot":     { "status": "no_data" },
          "Google-Extended":   { "status": "ok",
                                 "http_status": 200,
                                 "response_time_ms": 19,
                                 "blocked_by": null,
                                 "fetched_at": "2026-05-29T00:17:25Z" },
          "Bytespider":        { "status": "no_data" },
          "anthropic-ai":      { "status": "no_data" },
          "Applebot-Extended": { "status": "no_data" },
          "cohere-ai":         { "status": "no_data" }
        }
      }
    ],
    "pagination": { "limit": 50, "offset": 0, "total_pages": 1 }
  }
}

Failure responses

• 401 UNAUTHORIZED — API key missing or invalid (auth middleware).
• 404 crawl_not_found — crawl_id (UUID) does not match any crawl_jobs row.
• 403 not_workspace_member — caller is not a member of the crawl's owning workspace. Ownership check runs before the flag gate, so this surfaces even on workspaces where aeo.access is enabled.
• 403 feature_disabled — aeo.access flag is off for the workspace. Contact your workspace owner to enable.

Per-crawl per-agent robots.txt decisions for the nine canonical AI agents, with a regression diff against the immediately-prior crawl in the same workspace for the same domain. Surfaces what each AI agent is allowed or disallowed to crawl per the site's published robots.txt, and flags any agent that lost access since the prior crawl. The domain is derived from crawl_jobs.url via urlparse(...).netloc; robots.txt content is read from the crawler's robots_txt_cache.

Per-agent triple shape

Each entry in data.per_agent is keyed by the canonical preset name and carries a triple:

• current ("allow" | "disallow") — the agent's access decision per the crawl's robots.txt, evaluated at the site root.
• prior ("allow" | "disallow" | null) — the agent's decision per the prior crawl's robots.txt. null when no prior crawl exists for this workspace + domain.
• regression (boolean) — true only when prior == "allow" AND current == "disallow". The asymmetry is intentional — sites becoming more permissive are not regressions, and the field exists to surface AI-bot blocking that wasn't there before.

Token-only UAs — the real semantics

For the three robots.txt-only tokens (Google-Extended, anthropic-ai, Applebot-Extended) this endpoint is the source of truth for "is this AI vendor blocked." The matrix endpoint's synthetic HTTP probe rows for these tokens reflect what a probe with that user-agent sees; the robots-analysis output reflects what the AI vendors themselves see when reading robots.txt to decide whether to crawl. Token-only UAs are identified in data.token_only_agents for the frontend to render with the appropriate caveat.

{
  "success": true,
  "data": {
    "crawl_id": 11262,
    "workspace_id": 17,
    "domain": "example.com",
    "agents": [
      "GPTBot", "OAI-SearchBot", "ClaudeBot", "PerplexityBot",
      "Google-Extended", "Bytespider", "anthropic-ai",
      "Applebot-Extended", "cohere-ai"
    ],
    "token_only_agents": ["Applebot-Extended", "Google-Extended", "anthropic-ai"],
    "robots_txt_present": false,
    "prior_crawl_id": null,
    "prior_robots_txt_present": false,
    "per_agent": {
      "GPTBot":            { "current": "allow", "prior": null, "regression": false },
      "OAI-SearchBot":     { "current": "allow", "prior": null, "regression": false },
      "ClaudeBot":         { "current": "allow", "prior": null, "regression": false },
      "PerplexityBot":     { "current": "allow", "prior": null, "regression": false },
      "Google-Extended":   { "current": "allow", "prior": null, "regression": false },
      "Bytespider":        { "current": "allow", "prior": null, "regression": false },
      "anthropic-ai":      { "current": "allow", "prior": null, "regression": false },
      "Applebot-Extended": { "current": "allow", "prior": null, "regression": false },
      "cohere-ai":         { "current": "allow", "prior": null, "regression": false }
    }
  }
}

When no robots.txt is cached for the crawl's domain, every agent's current defaults to "allow" per RFC 9309 §2.2.1 / urllib.robotparser permissive semantics. robots_txt_present and prior_robots_txt_present let the UI render the appropriate state pane.

Failure responses

• 401 UNAUTHORIZED — API key missing or invalid (auth middleware).
• 404 crawl_not_found — crawl_id (UUID) does not match any crawl_jobs row.
• 403 not_workspace_member — caller is not a member of the crawl's owning workspace. Ownership check runs before the flag gate, so this surfaces even on workspaces where aeo.access is enabled.
• 403 feature_disabled — aeo.access flag is off for the workspace.

Per-(page, agent) detail powering the side-sliding view-as panel in the matrix UI. Surfaces the single agent's probe outcome for the single page (status / HTTP code / response time / response size / fetched-at) plus a 2 KB excerpt of the page's HTML and text content as captured during the crawl. Workspace ownership flows page→crawl→workspace: the page's parent pages.job_id is the integer FK to crawl_jobs.id, the parent crawl's UUID is resolved, and the standard resolve_crawl_access ownership-before-flag-gate sequence runs against the resolved workspace.

Path parameters

• page_id (integer, ≥1) — numeric pages.id.
• agent_name (string) — one of the nine canonical UA presets: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, anthropic-ai, Applebot-Extended, cohere-ai. Unknown names are rejected with 422 unknown_agent before any DB roundtrip.

Response shape

• data.access — the page_agent_access row for this (page, agent) pair, or {"status": "no_data"} when no row exists. Same status ENUM as the matrix endpoint plus the synthesized no_data. blocked_by is included in the envelope for forward-compatibility but is always null in current writes (reserved for a future writer-WU).
• data.content.html — the page's HTML content as captured during the crawl, truncated to 2048 characters.
• data.content.text — the page's extracted text content, truncated to 2048 characters.
• data.content.truncated (boolean) — true when either HTML or text exceeded the 2 KB excerpt limit. Lets the UI render a truncation badge.
• When no page_content row exists for the page, data.content is {"html": null, "text": null, "truncated": false}.

{
  "success": true,
  "data": {
    "page_id": 220605,
    "crawl_id": 11262,
    "workspace_id": 17,
    "agent": "GPTBot",
    "access": {
      "status": "ok",
      "http_status": 200,
      "response_time_ms": 27,
      "response_bytes": 4096,
      "blocked_by": null,
      "fetched_at": "2026-05-29T00:17:25Z"
    },
    "content": {
      "html": "",
      "text": "Example Domain. This domain is for use in illustrative examples ...",
      "truncated": false
    }
  }
}

Failure responses

• 401 UNAUTHORIZED — API key missing or invalid (auth middleware).
• 422 unknown_agent — agent_name is not in the nine-preset allowlist. Validated before any DB read.
• 404 page_not_found — no pages row with the given page_id.
• 403 not_workspace_member — caller is not a member of the workspace owning the page's parent crawl. Ownership check runs before the flag gate.
• 403 feature_disabled — aeo.access flag is off for the workspace.

Opt-in JS rendering probe for a single page. Runs an HTTP-only fetch (httpx, no JavaScript execution) and a full Playwright render (Chromium, JS-capable) for the supplied URL concurrently, then returns a byte-diff delta — signed char-count difference plus the top-N sentences present only in the Playwright render. Use it to surface content that AI crawlers without JS execution (the common case for HTTP-only crawlers like GPTBot and most archival bots) cannot see on the page. The probe is page-scoped: it runs once per call regardless of which matrix cell triggered it.

Wall-time is capped near 30s by running the two fetches concurrently (the operation surfaces a 30s timeout on each fetcher independently; total wall is the slower of the two). The endpoint is purely additive on the M1 single-UA Playwright flow — the default crawl behavior is not modified.

Path parameters

• crawl_id (string, UUID) — the crawl_jobs.job_id UUID. Same shape as /{crawl_id}/matrix and /{crawl_id}/robots-analysis.

Request body

{
  "page_id": 220605,
  "url": "https://example.com/"
}

• page_id (integer, ≥1) — numeric pages.id; the page must belong to the crawl identified by crawl_id.
• url (string, 1–2048 chars) — must equal pages.url for the supplied page_id. This is a defense-in-depth check that prevents the endpoint from acting as a general-purpose probe for arbitrary URLs — the client must demonstrate they already discovered the URL via a crawl owned by the requesting user's workspace.

Response shape

• data.probe.http_status / data.probe.playwright_status — HTTP status codes from each fetcher (null if the fetcher errored before receiving a response).
• data.probe.http_fetch_ms / data.probe.playwright_render_ms — wall-clock time per fetcher in milliseconds.
• data.probe.http_error / data.probe.playwright_error — non-null when that fetcher raised; the other fetcher's results are still returned (failures are partial, not fatal).
• data.delta.char_count_delta — signed integer (positive when Playwright sees more content than HTTP-only).
• data.delta.http_char_count / data.delta.playwright_char_count — visible-text character counts per side, after stripping <script> / <style> / <noscript> / <template>.
• data.delta.csr_only_sentence_count — total count of sentences present in the Playwright render but absent from the HTTP-only fetch (after a 30-char floor to drop nav/menu noise).
• data.delta.csr_only_sentences — up to 20 of those sentences, each truncated to 200 characters. List order is the order they appear in the Playwright body. Empty array when the page is fully visible to HTTP-only crawlers.

{
  "success": true,
  "data": {
    "crawl_id": 11266,
    "workspace_id": 17,
    "page_id": 220605,
    "url": "https://example.com/",
    "probe": {
      "http_status": 200,
      "http_fetch_ms": 25,
      "http_error": null,
      "playwright_status": 200,
      "playwright_render_ms": 835,
      "playwright_error": null
    },
    "delta": {
      "char_count_delta": 0,
      "http_char_count": 230,
      "playwright_char_count": 230,
      "csr_only_sentence_count": 0,
      "csr_only_sentences": []
    }
  }
}

The example above shows a static-HTML target (example.com): the HTTP-only fetch and the Playwright render produce identical visible text, so the delta is zero and no CSR-only sentences are reported — that is the contract working correctly, not an error. CSR-heavy targets (e.g., a React SPA without server-side rendering) produce non-zero char_count_delta and a populated csr_only_sentences list.

Failure responses

• 401 UNAUTHORIZED — API key missing or invalid (auth middleware).
• 404 crawl_not_found — crawl_id (UUID) does not match any crawl_jobs row.
• 403 not_workspace_member — caller is not a member of the crawl's owning workspace. Ownership check runs before the flag gate.
• 403 feature_disabled — aeo.access flag is off for the workspace.
• 422 page_not_in_crawl — page_id does not exist, or exists but belongs to a different crawl. The endpoint cannot probe pages outside the supplied crawl.
• 422 url_mismatch — url in the body does not match pages.url for the supplied page_id. This is the defense-in-depth check that gates the endpoint against arbitrary-URL probing.
• 500 probe_failed — an unhandled exception inside the probe execution (distinct from per-fetcher errors, which surface as non-null http_error / playwright_error in a 200 response).

List WordPress connections.

Register new WordPress site.

Update connection settings.

Disconnect WordPress site.

Test connectivity.

Test connection before saving.

Get sync history.

Push pages/posts from WordPress.

Push navigation menus from WordPress.

Push categories/tags from WordPress.

Pull content from WordPress.

Push navigation to WordPress.

Push taxonomy to WordPress.

Push pages to WordPress.

Test MCP tool on WordPress.

Get available pricing plans.

Get current subscription details.

Create Stripe Checkout session.

Change subscription plan (upgrade/downgrade).

Cancel subscription at end of period.

Undo pending cancellation.

Get Stripe invoices.

Open Stripe Customer Portal.

Get current billing period usage.

Get usage history across periods.

Get spending cap status.

Set or update spending cap.

List available credit bundles.

Get credit balance.

Purchase credits via Stripe.

Get credit purchase history.

Estimate costs for current period.

Get current subscription tier.

Get usage quota and limits.

Check downgrade eligibility.

List integration status.

Generate Google OAuth consent URL.

OAuth callback (token exchange).

Save OAuth tokens.

Disconnect integration.

Fetch GA + GSC data for URLs.

List GA4 properties.

List Search Console sites.

Save selected GA/GSC property.