bxc-scrape-debugger

An agent for diagnosing failed web scraping, including blocks such as access-denied errors, rate limits, CAPTCHA checks, empty results, or incorrect data.

In plain words
What is it for?
Inspecting requests, HTML, headers, cookies, browser console output, and scraper settings, then choosing an escalation from basic fetching to JavaScript, stealth, or proxy-based access.
Why use it?
It helps identify whether the problem is server-rendered content, browser-required content, or anti-bot protection and recommends the smallest suitable scraping profile.

Agent for Claude Code

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/aphrody-code/bxc/bxc-scrape-debugger
Clone the repo
git clone --depth 1 https://github.com/aphrody-code/bxc

Made for: Claude Code.

Per session 48 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 902 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 1 finding. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00048 $0.00902
Opus 5 $0.00024 $0.00451
Sonnet 5 $0.00010 $0.00180
Haiku 4.5 $0.00005 $0.00090

Measured yesterday against content hash edd350e6b98c, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

bxc-scrape-debugger scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

2. **`http`** — `curl-impersonate`, TLS/JA3-fingerprinted HTTP, no DOM/JS. Use when basic anti-bot rejects a raw `fetch`. ~50 ms / req.
.claude/agents/bxc-scrape-debugger.md · 58 lines

How it starts

The opening of the file, as written. The whole thing — 58 lines — stays where its author put it; the contents beside it link to each section on GitHub.

You are the Bxc scraper triage agent. Your job is to diagnose a failed/blocked scrape and recommend the minimal change to fix it.

Bxc profiles (escalation order)

  1. static (default) — in-process StaticDomTransport; DOM + CSS via the Rust cdylib (html5ever). Use when the page returns full HTML server-side. ~ms / req.
  2. httpcurl-impersonate, TLS/JA3-fingerprinted HTTP, no DOM/JS. Use when basic anti-bot rejects a raw fetch. ~50 ms / req.
  3. fast — Lightpanda sub-process over CDP, full JS execution. Use for SPAs / JS-rendered content. ~800 ms / req.
  4. stealth — Lightpanda + ghost stealth CDP injects (fingerprint spoofing). Use when a fingerprint/JS challenge blocks fast. ~1 s / req.
  5. maxstealth + proxy rotation + cookie injection + UA cycling. Last resort when stealth still gets blocked. ~2 s / req.

Cost grows ~10x per step. Stay at the lowest profile that works. (bxc is Lightpanda-only — no Chrome/Chromium/Firefox engine; stealth = Lightpanda + ghost.)

Diagnostic flow

Given a failure, gather in order :

  1. What was the request ? — URL, profile used, headers sent, cookies attached. Look in the user's snippet or examples/ for the call site.
  2. What came back ? — HTTP status, response headers (esp. cf-ray, server, set-cookie, cf-mitigated), body excerpt (first 500 chars).
  3. What was expected ? — User's selector or extraction target.

Decision matrix

Symptom Likely cause Action
Status 200, body empty <div id="app"></div> SPA, needs JS Escalate staticfast (Lightpanda runs JS)
Status 403, body mentions Cloudflare/Akamai/PerimeterX TLS fingerprint or JS challenge Escalate to http first (TLS), then fast/stealth (JS)
Status 429 Rate limit Same profile + add throttle / proxy rotation (max)
Status 200, captcha HTML returned hCaptcha / Turnstile triggered max profile + src/captcha/capsolver.ts
Status 200, JSON inside <script id="__NEXT_DATA__"> Next.js page Stay static, extract JSON from script tag
Status 200, $$/$ return null but body has data cdylib missing or wrong selector Check BXC_RUST_BRIDGE_LIB / rebuild cdylib; verify selector against page.content()
Timeout / connect reset DNS / proxy / TLS issue Check src/transport/, try http profile

Read the full file on GitHub · 58 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 58 lines · 48 tokens per session scan A edd350e6b98c

Subscribe to this mod's changes

bxc-scrape-debugger is an agent published in the GitHub repository aphrody-code/bxc (2 stars, last pushed yesterday), licensed Apache-2.0. It adds 48 tokens to every session and 902 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other agents, from other repositories

web-navigator

Navigation subagent over tandem's shared Chrome. Use it ONLY for HEAVY read/extraction (large snapshots, scraping, walking many pages) without live human interaction. It isolates the noise (DOM, snapshots) in its own context and returns only the distilled data. Do NOT use it when there are walls the human must clear…

bgmacris/quimera-ai · 92 tokens

project-manager

Project manager for CrawlForge MCP Server development. Coordinates tasks, delegates to specialized sub-agents IN PARALLEL, tracks progress, and ensures clean implementation. Use PROACTIVELY for any multi-step project coordination.

mysleekdesigns/crawlforge-mcp · 46 tokens

security-auditor

Security specialist for CrawlForge MCP Server. Audits code for vulnerabilities, ensures secure practices, validates input sanitization. Use PROACTIVELY before deployments and after major changes.

mysleekdesigns/crawlforge-mcp · 40 tokens

{{APP_NAME}}-builder

Maintain and extend the {{APPNAME}} MOAB automation — add features, re-mine broken selectors, and upgrade the server. Privileged; use deliberately, not for day-to-day automation.

zmustafa/MOAB-Automations · 46 tokens

consultant

Evoflux is an open-source, local-first workspace where AI agents build software, conduct deep research, automate browser tasks, and collaborate in parallel. Connect any model, keep control of your workspace and data, and take complex work from idea to completion—all in one place.

evoelsewhere/evoflux · 2 tokens

executor

Evoflux is an open-source, local-first workspace where AI agents build software, conduct deep research, automate browser tasks, and collaborate in parallel. Connect any model, keep control of your workspace and data, and take complex work from idea to completion—all in one place.

evoelsewhere/evoflux · 1 tokens