context-builder: Instructions file for Codex

AGENTS.md - AI Agent Instructions

This file helps AI agents quickly understand and contribute to the Context Builder codebase.

Project Overview

Context Builder is a blazing-fast Rust CLI for aggregating entire codebases into single, LLM-friendly markdown files. Published on crates.io under MIT license.

If this is your first time: Read this file, then run cargo run -- --help to see all options.

---

Deep Think Authority Mode (optional handoffs)

When the user points at docs/handoffs/** or an AUTHORITY document:

1. Follow Authority Mode rules in docs/agent-harness/antigravity-system-prompt.md.
2. AUTHORITY is law for implementation; do not redesign.
3. If the live tree contradicts AUTHORITY, stop with a CONFLICT packet (docs/agent-harness/templates/05-conflict.md).
4. Prefer tools on the real repo; do not demand a re-dump of the full context-builder snapshot unless tools are unavailable.
5. End with a RESULT block (docs/agent-harness/templates/04-result.md).

Pipeline overview: docs/agent-harness/README.md. Skill entrypoint: SKILL.md.

---

Tech Stack

Technology	Usage
Language	Rust (Edition 2024)
Build	Cargo (no npm/bun/node)
CLI	clap (derive)
Parallelism	rayon (optional, default on) + crossbeam-channel
Diffing	similar (unified diffs)
File traversal	ignore crate (gitignore-aware)
Token counting	tiktoken-rs (cl100k_base)
Caching	JSON snapshots + std::fs::File advisory locking (fs2 removed in v0.9.0)
Config	TOML (context-builder.toml)
Encoding	encoding_rs (transcoding non-UTF-8)
Logging	env_logger
Branch	master (not main)

---

Project Structure

context-builder/
├── src/
│   ├── main.rs              # Entry point — calls lib::run()
│   ├── lib.rs               # Core orchestration, run_with_args(), Prompter trait, --init
│   ├── cli.rs               # Args struct via clap derive
│   ├── config.rs            # Config struct, TOML deserialization
│   ├── config_resolver.rs   # Merges CLI args + TOML config (CLI > config > defaults)
│   ├── file_utils.rs        # .gitignore-aware traversal, OverrideBuilder for custom ignores
│   ├── tree.rs              # BTreeMap file tree (deterministic ordering)
│   ├── state.rs             # ProjectState/FileState structured snapshots
│   ├── markdown.rs          # Streaming file renderer, binary detection, encoding, parallel
│   ├── cache.rs             # JSON-based caching with std::fs::File locking, old cache migration
│   ├── diff.rs              # Per-file unified diffs via similar
│   ├── token_count.rs       # Real tokenization via tiktoken-rs (o200k/cl100k, lazy init)
│   ├── languages.rs         # Canonical extension → code-fence language map (v0.10)
│   └── tree_sitter/         # AST signatures/structure/smart-truncation (feature-gated)
│       ├── language_support.rs  # Signature/Visibility types, LanguageSupport trait
│       ├── signatures.rs        # extract_signatures + markdown formatting
│       ├── structure.rs         # structural summaries
│       ├── truncation.rs        # AST-boundary smart truncation (char-safe per B19)
│       ├── mod.rs               # feature-gated facade (is_supported_extension, etc.)
│       └── languages/           # per-language extractors: rust, javascript, typescript,
│                                #   python, go, java, c, cpp
├── tests/                   # 10 integration test files
├── benches/                 # Criterion benchmark suite
├── scripts/                 # generate_samples.rs (benchmark dataset generator)
├── context-builder.toml     # Project's own config file
├── Cargo.toml               # Crate metadata, dependencies, features
├── DEVELOPMENT.md           # Contributor guide
├── BENCHMARKS.md            # Performance benchmarking guide
├── CHANGELOG.md             # Release history
└── .github/workflows/ci.yml # CI: fmt, clippy, build, test, security audit (ubuntu/win/macos)