s-fetch

s-fetch

Fetches a URL directly via scrapling, bypassing bot detection.

Setup

If the bash command below fails because scrapling-fetch-mcp isn't installed (e.g. No such file or directory for the bin/python3 path), follow references/install.md once before proceeding. If it fails with Executable doesn't exist (browser binaries out of date after an upgrade), run only step 2 of that file.

Arguments

$ARGUMENTS — parse as <url> [mode] [format]

• url (required)
• mode (optional, default basic): basic | stealth | max-stealth
• format (optional, default markdown): markdown | html

Instructions

Run the following Bash command, substituting URL, MODE, and FORMAT with the parsed values:

"$(uv tool dir)/scrapling-fetch-mcp/bin/python3" - <<'PYEOF'
import asyncio
from contextlib import redirect_stdout
from os import devnull

async def fetch(url, mode):
    with open(devnull, "w") as null, redirect_stdout(null):
        from scrapling.fetchers import AsyncFetcher, StealthyFetcher
        if mode == "basic":
            return await AsyncFetcher.get(url, stealthy_headers=True)
        elif mode == "stealth":
            return await StealthyFetcher.async_fetch(url, headless=True, network_idle=True)
        elif mode == "max-stealth":
            return await StealthyFetcher.async_fetch(
                url, headless=True, block_webrtc=True,
                network_idle=True, disable_resources=False, block_images=False
            )
        else:
            raise ValueError(f"Unknown mode: {mode}")

url = "URL"
mode = "MODE"
fmt = "FORMAT"

page = asyncio.run(fetch(url, mode))
html = page.html_content

if fmt == "markdown":
    from bs4 import BeautifulSoup
    from markdownify import markdownify as md
    soup = BeautifulSoup(html, "lxml")
    for tag in soup(["script", "style"]):
        tag.extract()
    body = soup.find("body")
    print(md(str(body if body else soup)))
else:
    print(html)
PYEOF

If mode was not given and basic returns blocked or empty content, retry with stealth, then max-stealth.