Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add instructions/nolindnaidoo/scrape-le/agents-mdgit clone --depth 1 https://github.com/nolindnaidoo/scrape-leWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/instructions/nolindnaidoo/scrape-le/agents-md)<a href="https://agentmods.dev/instructions/nolindnaidoo/scrape-le/agents-md"><img src="https://agentmods.dev/badge/instructions/nolindnaidoo/scrape-le/agents-md.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.05501 | $0.05501 |
| Opus 5 | $0.02750 | $0.02750 |
| Sonnet 5 | $0.01100 | $0.01100 |
| Haiku 4.5 | $0.00550 | $0.00550 |
Grade A, and why
scrape-le AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
This is a copy
83% identical to regex-le AGENTS.md — 97 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 342 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AGENTS.md — Scrape-LE
Technical source of truth for this repo. README.md is the user-facing doc; this file is for anyone (human or agent) changing the code.
This repo hosts two products: the extension at the root (this document's scope) and the Rust CLI in crate/ (its own AGENTS.md + SPEC.md). The shared detection corpus lives at signatures/ and fixtures/; scripts/check-signature-parity.ts fails CI when this extension drifts from it.
What this is
A VS Code extension that loads a URL in headless Chromium (playwright-core) and reports what would block a scraper: HTTP status/title/load time, console errors, a full-page screenshot, anti-bot vendor detection, rate-limit headers, robots.txt evaluation, and authentication walls. Network access is scoped to the URL under check plus that origin's /robots.txt — nothing else, ever.
Architecture
extension.ts activate(): createNotifier() + createStatusBar() -> registerCommands()
commands/ one file per command; deps injected as a frozen bag
check.ts checkUrl prompt + executeCheck orchestration (progress, status bar, output)
checkSelection.ts extracts a URL from the selection, reuses executeCheck
setup.ts browser install/verify quick pick
help.ts opens the help markdown document
scraper/
browser.ts chromium.launch wrapper (headless, hardened args)
checker.ts page load ('load' + best-effort 5s networkidle), console capture,
screenshot (format/quality), runs detections
install.ts async browser install via bundled playwright-core CLI in a child
Node process (ELECTRON_RUN_AS_NODE); manual command pinned to the
shipped playwright-core version
detectors/
heuristics.ts THE vendor signature table + single-evaluate page probe
antibot.ts header pass + one page.evaluate for all vendors
ratelimit.ts X-RateLimit / RateLimit / Retry-After headers + HTTP 429
robotstxt.ts RFC 9309 groups, Allow/Disallow longest-match, * and $ patterns
authentication.ts 401/403, login forms, keywords, path-segment URL check
index.ts runDetections: enabled detectors in parallel, failures isolated
ui/ notifier (notificationsLevel re-read per call: all -> everything,
important -> warn+error, silent -> error only), statusBar (owns
visibility, follows statusBar.enabled live), output channel
config/config.ts getConfiguration() snapshot; DEFAULT_CONFIG table
utils/ url (validate/normalize/extract/filename), errorHandling
(sanitizeErrorMessage wired into user-facing errors)
types.ts shared types only — no logic
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 342 lines · 5,501 tokens per session scan A 3a951f0a4cb3
scrape-le AGENTS.md is an instructions file published in the GitHub repository nolindnaidoo/scrape-le (1 stars, last pushed 4d ago), licensed MIT. It adds 5,501 tokens to every session, about $0.0275 per session on Opus 5. A static security scan graded it A with 0 findings. It is 83% identical to regex-le AGENTS.md, differing in 97 lines, and is treated as a copy.
Other instructions, from other repositories
obscura AGENTS.md
AGENTS.md instructions for h4ckf0r0day/obscura, covering agents.md, build, rendering and stealth, no rendering, with rustls or stealth and test.
ark CLAUDE.md
Claude Code instructions for chakra-ui/ark, covering ark ui - claude development guide, documentation structure, quick start, common commands and install dependencies.
serve-sim AGENTS.md
Instructions for EvanBacon/serve-sim, covering native build notes, e2e testing with agent-browser and e2e testing via the serve-sim cli.
bot-signal AGENTS.md
AGENTS.md instructions for okasi/bot-signal, covering agents.md, project overview, repository layout, commands and conventions.
weaverse AGENTS.md
AGENTS.md instructions for Weaverse/weaverse, covering agents.md - weaverse sdks development guide, project overview, quick reference, single package commands and monorepo architecture.
weaverse copilot-instructions.md
Copilot instructions for Weaverse/weaverse, covering copilot instructions, coding standards, typescript guidelines, react guidelines and error handling.