Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add instructions/bda-research/node-crawler/agents-mdgit clone --depth 1 https://github.com/bda-research/node-crawlerWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00408 | $0.00408 |
| Opus 5 | $0.00204 | $0.00204 |
| Sonnet 5 | $0.00082 | $0.00082 |
| Haiku 4.5 | $0.00041 | $0.00041 |
Grade A, and why
node-crawler AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
What it actually says
AGENTS.md
This file provides guidance to the AI agent when working with code in this repository.
Build & Test
Tests run against compiled output in dist/, not source. Always build before testing:
npm run build # tsup → dist/index.js + dist/index.cjs
npm test # NODE_ENV=test ava (runs test/*.js, excludes test/*test.js)
CI pipeline: npx eslint && npm run build && npm test
Testing
- ava with nock for HTTP mocking. No real network calls.
- Test files are plain
.js(not.ts) and import from../dist/index.js. - Ava config excludes
test/*test.js— new test files should NOT use the.test.jssuffix (e.g.callback.js, notcallback.test.js). - Tests use a custom
testCbwrapper fromtest/lib/avaTestCb.jsfor callback-based async. Callt.end()to signal completion. - Default test timeout is 20s.
Code Style
- TypeScript strict mode, ESM throughout (
"type": "module"). - ESLint enforced on
src/**/*.ts: double quotes, semicolons required,no-consoleis an error, unused vars prefixed with_are allowed. - Target: ES2020. Module resolution: bundler.
Architecture Notes
gotis lazy-loaded on first use via dynamicimport("got")— do not add a top-level static import for it.cheeriois the default HTML parser (exposed asres.$). jsdom is not supported.- Rate limiter uses a cluster of independent limiters.
rateLimitis global-only — modify per-limiter viacrawler.setLimiter(), not per-request options. - The
callbackpattern requires callingdone()to signal task completion and release the slot.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 36 lines · 408 tokens per session scan A 99682743a495
node-crawler AGENTS.md is an instructions file published in the GitHub repository bda-research/node-crawler (6,798 stars, last pushed 2mo ago), licensed MIT. It adds 408 tokens to every session, about $0.0020 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other instructions, from other repositories
spider CLAUDE.md
Instructions for spider-rs/spider, covering spider - development guide, quick reference, getting started, spider cloud (recommended) and local crawl (no api key).
boss-zhipin-scraper AGENTS.md
Instructions for eatmoreduck/boss-zhipin-scraper, covering agents.md, 这是什么, 目录结构, 环境与命令 and 改代码时的硬规则.
AfdianToMarkdown CLAUDE.md
Claude Code instructions for PhiFever/AfdianToMarkdown, covering claude.md, 项目概述, 常用命令, 构建 and 运行（示例）.
deepspider CLAUDE.md
Claude Code instructions for ma-pony/deepspider, covering deepspider repository guidance, runtime boundaries, reverse-engineering contract, tool schemas and commands.
qiankun AGENTS.md
Instructions for umijs/qiankun, covering qiankun project knowledge base, structure, architecture (big picture), commands and build (vite 8 → package javascript; tsc → declarations).
joint CLAUDE.md
Instructions for clientIO/joint, covering claude.md, project overview, common commands, install dependencies and build distribution files (runs grunt).