crawler instructions

31 tagged crawler, measured the same way as everything else here.

dotnetpower/infomesh

Instructions file GitHub Copilot

Instructions for dotnetpower/infomesh, covering infomesh — copilot instructions, project overview, core principles, tech stack and project structure.

5 3mo ago A 13,682 tokens original MIT

mysleekdesigns/crawlforge-mcp

Instructions file

Instructions for mysleekdesigns/crawlforge-mcp, covering claude.md, project overview, development commands, install dependencies and setup (required for first run - users only).

2 2d ago C 4,166 tokens original MIT

bxc CLAUDE.md

27

aphrody-code/bxc

Instructions file

Instructions for aphrody-code/bxc, covering claude.md — bxc, objectif directeur (depuis 2026-08-27), rappels critiques, commandes essentielles and commandes des scrapers dédiés.

2 2d ago B 4,785 tokens original Apache-2.0

bxc GEMINI.md

28

aphrody-code/bxc

Instructions file Gemini CLI

Instructions for aphrody-code/bxc: This repository's operating guide is CLAUDE.md — read it first.

2 2d ago A 91 tokens original Apache-2.0

codex-seo AGENTS.md

29

CaffNoir59/codex-seo

Instructions file CodexOpenCode

AGENTS.md instructions for CaffNoir59/codex-seo: Use this repository as a native Codex SEO framework. Prefer project commands over ad hoc scripts.

1 1mo ago A 120 tokens

auxiliar-ai/auxiliar-mcp

Instructions file

Claude Code instructions for auxiliar-ai/auxiliar-mcp, covering what this is, commands, architecture (small, deliberately), the four tools and non-obvious constraints.

0 1mo ago C 1,504 tokens original MIT archived

wyattowalsh/mcp-crawl4ai

Instructions file CodexOpenCode

AGENTS.md instructions for wyattowalsh/mcp-crawl4ai, covering agents.md -- mcp-crawl4ai, stack, key files, commands and install.

0 4mo ago A 1,254 tokens original MIT