har-derived-api-client

HAR-Derived API Client

Drive a website once with a real browser while recording its network traffic to a HAR file, then distill that HAR into the site's private JSON API so you can call it directly with plain HTTP — far cheaper and faster than browser-controlling the page on every request. Credit: trick by Jared Longster, popularized by Dax (thdxr). This captures and replays; it does NOT bypass auth, solve CAPTCHAs, or defeat bot-detection — if the site needs a logged-in session, you carry its headers/cookies forward, you don't forge them.

The scripts are stdlib-plus-Playwright: capture needs Playwright, derivation is pure stdlib, replay needs only requests/httpx (or curl).

Covers every Hermes browser pathway: the default local browser_navigate backend, plus the cloud/remote backends (Browserbase, Browser-Use, Firecrawl) and any /browser connect CDP endpoint. There are two capture scripts — one for a browser you launch, one for a browser you attach to over CDP — because HAR recording works differently in each case (see How to Run).

When to Use

• "Build a CLI/client for " — derive its API instead of scripting clicks.
• "This site has no public API but the page clearly fetches JSON."
• You're about to loop browser_navigate for the same query repeatedly — stop and derive the endpoint once.
• Reverse-engineering an autocomplete, search, feed, or checkout XHR.
• You captured a session on a cloud backend (Browserbase / Browser-Use / Firecrawl) or via /browser connect and want the API without re-renting the browser.

Prerequisites

• Playwright + a browser binary (capture step only):
  • pip install playwright then playwright install chromium
  • (If a system Playwright already has browsers under ~/.cache/ms-playwright, reuse it.)
• requests or httpx for the replay step (stdlib urllib also works).
• No API keys. Any keys/tokens the client needs are the ones the HAR captured.
• For the CDP path (har_capture_cdp.py): a reachable CDP endpoint. On Hermes, run /browser connect to print the active endpoint, or read BROWSER_CDP_URL / browser.cdp_url in config. Cloud backends expose it as cdpUrl/connectUrl.