crawler instructions

firecrawl/firecrawl

Instructions file CodexOpenCode

AGENTS.md instructions for firecrawl/firecrawl: Firecrawl is a web scraper API. The directory you have access to is a monorepo.

firecrawl/firecrawl

Instructions file

Claude Code instructions for firecrawl/firecrawl: Firecrawl is a web scraper API. The directory you have access to is a monorepo.

TeamWiseFlow/xiaobei

Instructions file CodexOpenCode

A set of AGENTS.md project instructions for the TeamWiseFlow/xiaobei repository. It covers Docker deployment rules, crew template development, skill files, and command-line guidance.

TeamWiseFlow/xiaobei

Instructions file

A set of Claude Code project instructions for the TeamWiseFlow/xiaobei repository. It covers CLAUDE.md, Docker deployment rules, crew template development, skill files, and command-line guidance.

bda-research/node-crawler

Instructions file CodexOpenCode

Instructions for bda-research/node-crawler, covering agents.md, build & test, testing, code style and architecture notes.

spider-rs/spider

Instructions file

Instructions for spider-rs/spider, covering spider - development guide, quick reference, getting started, spider cloud (recommended) and local crawl (no api key).

eatmoreduck/boss-zhipin-scraper

Instructions file CodexOpenCode

Project instructions for a Python scraper that connects to a user's already logged-in Chrome browser and collects public job listings from BOSS直聘, a Chinese job site. They describe the files, setup, commands, and rules for changing the code.

codelibs/fess

Instructions file

Instructions for codelibs/fess, covering claude.md, project overview, tech stack, development commands and setup.

janreges/siteone-crawler

Instructions file

Instructions for janreges/siteone-crawler, covering claude.md, core working principles, setup after clone, build & test commands and quick run.

us/crw

Instructions file CodexOpenCode

AGENTS.md instructions for us/crw, covering agents.md, two-minute system map, source-of-truth matrix, if you are changing x and verification.

djfksjd/ir-search

Instructions file CodexOpenCode

Repository instructions for an ir-search plugin that surveys South Korean government and public-sector support programmes. They explain how the crawler works, where its scripts are, and which access and rate-limit rules apply.

just-every/mcp-read-website-fast

Instructions file

Instructions for just-every/mcp-read-website-fast, covering claude.md, project overview, core modules & files, commands and development.

alex-on-ai/WebReaper

Instructions file

Instructions for alex-on-ai/WebReaper, covering claude.md, what this is, commands, test reality and toolchain.

PhiFever/AfdianToMarkdown

Instructions file

Claude Code instructions for PhiFever/AfdianToMarkdown, covering claude.md, 项目概述, 常用命令, 构建 and 运行（示例）.

supadata-ai/mcp

Instructions file

Instructions for supadata-ai/mcp, covering claude.md, project overview, key commands, development commands and single test execution.

Praeviso/crawl4weibo

Instructions file CodexOpenCode

Instructions for Praeviso/crawl4weibo, covering repository guidelines, project structure & module organization, build, test, and development commands, coding style & naming conventions and testing guidelines.

Praeviso/crawl4weibo

Instructions file

Instructions for Praeviso/crawl4weibo, covering claude.md, project overview, development commands, setup and installation and testing.

tristanwqy/GooFish-AIMonitor

Instructions file CodexOpenCode

Repository guidance for GooFish-AIMonitor, a web console for finding products on Xianyu and monitoring price drops. It documents the Python and JavaScript technologies, project rules, validation commands, and code structure.

pixlie/SmartCrawler

Instructions file

Claude Code instructions for pixlie/SmartCrawler, covering claude.md, development workflow, core commands, build and run and build the project.

ma-pony/deepspider

Instructions file

Claude Code instructions for ma-pony/deepspider, covering deepspider repository guidance, runtime boundaries, reverse-engineering contract, tool schemas and commands.

ForkPoint/agent-lighthouse

Instructions file CodexOpenCode

Instructions for ForkPoint/agent-lighthouse, covering working in this repository, layout, commands, one audit, one file, one dossier and the meta law.

ForkPoint/agent-lighthouse

Instructions file

Instructions for ForkPoint/agent-lighthouse, a project described as: Lighthouse-style audits for AI-agent readiness, LLM crawlers, MCP clients, and the agentic web.

Smart75850/smart-agent

Instructions file

A set of Claude Code instructions for the smart-agent project, an AI-agent configuration project. It explains how to make small, low-cost changes first and clearly mark uncertainty.

dkyazzentwatwa/threads-mcp-crawler

Instructions file

Instructions for dkyazzentwatwa/threads-mcp-crawler, covering threads scraper mcp - developer guide, architecture overview, design philosophy, tech stack and module responsibilities.