crawlforge-stealth-browsing

CrawlForge Stealth Browsing

Get past bot-detection systems, and request pages in a given locale. Use stealth_mode when a normal scrape is blocked, and localization to look up the locale values (Accept-Language, timezone, currency) of a country, which you then pass to fetch_url or stealth_mode yourself.

When to escalate to stealth_mode

Escalate from a normal scrape / fetch_url (see crawlforge-web-scraping) when:

• The site returns 403 or 429 on a regular fetch.
• You get a CAPTCHA or a "please enable JavaScript" interstitial.
• Content comes back empty or only a shell (JS-rendered SPA).
• The site uses Cloudflare, DataDome, PerimeterX, or similar protection.

stealth_mode drives a real browser with randomized fingerprints, human behavior simulation, and WebRTC/canvas/WebGL spoofing.

When you already know the site blocks, scrape with escalate: true does both steps in one call: the plain fetch first, then the same stealth browser only if that fetch is walled. Projected at 7, charged 2 when the plain fetch worked. Still never reach for stealth_mode first.

stealth_mode (cost: 5)

stealth_mode is operation-based. operation:"scrape" is the one-shot path: it renders one URL and returns the requested formats (examples below).

create_context then create_page is a navigation and challenge check, not a multi-step path: create_page loads urlToTest, reports its status, title and any challenge page it detected, and closes the page before the call returns. Nothing on that page carries over to a later call.

{
  "tool": "stealth_mode",
  "params": { "operation": "create_page", "contextId": "<id-from-create_context>", "urlToTest": "https://protected-site.com" }
}

For work over several calls on one page (log in, then read; click through a flow), use browser_session with stealth:true (see the crawlforge-browser-sessions skill).

Operations: scrape, configure, create_context, create_page, get_stats, cleanup. stealthConfig.level is basic / medium (default) / advanced. Always run cleanup when done to release the browser.