bda-research

bda-research/node-crawler

Instructions file CodexOpenCode

Instructions for bda-research/node-crawler, covering agents.md, build & test, testing, code style and architecture notes.

bda-research/node-crawler

Skill Claude CodeCodex

Node.js web crawler for production-grade, large-scale tasks — NOT for simple one-off requests. Use only when: bulk scraping, batch downloading, multi-page crawling, long-running spiders, or complex multi-step workflows that need connection pooling, rate limiting, proxy rotation, automatic retries, and Cheerio parsing.…