agentmods.dev

spider-rs/spider

Foundational low latency web data collecting in Rust

About the project

Spider is a Rust web-crawling and scraping engine that fetches pages, follows links, and uses a headless browser when JavaScript rendering is needed. It is for collecting web data locally or through Spider Cloud, from individual scripts to distributed workers.

Latest release v2.53.9 · 30 Oct 2026

1 file for Claude Code: spider CLAUDE.md — 4,676 tokens loaded in every session.

Open on GitHub spider.cloud
2,833Stars on the repository
1Files it configures its agents with
4,676Tokens loaded in every session
1Agent configured

Instructions

These files are spider-rs/spider's own configuration — they tell Claude Code how to work on this repository, so they are not mods to install elsewhere. Copy one as a starting point and replace the parts that are about this project.