torchtalk: Instructions file for Codex

AGENTS.md

Instructions for any coding agent working in this repository. Claude Code, Codex, Cursor, Gemini CLI and Copilot all read this file (see docs/agent-setup.md for how each tool picks it up and registers the MCP server).

What this project is

TorchTalk is an MCP server that answers structural questions about PyTorch and PyTorch-based codebases with file and line evidence. It indexes a source checkout once, caches the result, and exposes seven tools (get_status, trace, search, graph, modules, tests, affected) over MCP. Framework conventions live in TOML manifests ("harnesses"), not in code.

Commands

# Install
pip install -e ".[dev]"

# Test (what CI runs)
python -m pytest tests/ -v --cov=torchtalk --cov-report=term-missing
PYTORCH_SOURCE=/path/to/pytorch pytest tests/test_binding_detector_pytorch.py

# Lint and format (what CI runs)
ruff check src/torchtalk tests
ruff format --check src/torchtalk tests

# Harness smoke test against a real checkout
python scripts/harness_smoke.py --harness pytorch --source /path/to/pytorch

# Build the index, then serve
torchtalk index build --source /path/to/pytorch
torchtalk mcp-serve --source /path/to/pytorch

--harness <name> selects a manifest anywhere a source is indexed or served. A repo can ship its own .torchtalk.toml to activate a harness automatically.

Project structure

src/torchtalk/
├── server.py                # FastMCP app: the 7 tool definitions and run_server
├── cli.py                   # CLI: init, status, index, mcp-serve, snapshot, cursor-add
├── indexer.py               # Index build, cache load/save, incremental update, _state
├── harness.py               # ConventionManifest: TOML loading, extends, registry
├── manifests/               # Built-in harnesses: pytorch, torch-extension, vllm, torchvision
├── config.py                # Source resolution (flag, env, config file, .torchtalk.toml)
├── snapshots.py             # Named index snapshots: save/load/diff/export/import
├── symbols.py               # Package-qualified symbol ids for cross-repo merging
├── integration_manifest.py  # Strict loader for tests/integration/*.yml anchors
├── formatting.py            # CompactText / Markdown response formatters
├── tools/                   # Mode handlers behind each MCP tool
│   ├── ops.py               # trace, search
│   ├── graph.py             # graph (callers / calls / impact)
│   ├── modules.py           # modules
│   ├── tests.py             # tests
│   ├── affected.py          # affected
│   └── common.py
└── analysis/
    ├── binding_detector.py  # pybind11 / TORCH_LIBRARY detection (tree-sitter)
    ├── cpp_call_graph.py    # C++ call graph extraction (libclang, parallel)
    ├── libclang_env.py      # libclang discovery and version check
    ├── python_analyzer.py   # Python module/class analysis (AST)
    ├── extractors.py        # Manifest-driven registration extractors (Python AST)
    ├── external_refs.py     # ExternalRef edges: imports into depends_on harnesses
    ├── affected.py          # Changed C++ funcs -> impacted Python tests
    ├── alias_map.py         # Python call -> C++ symbol aliases (native_functions.yaml)
    ├── backward_bridge.py   # Backward ATen functions -> forward ops
    ├── decomp_aliases.py    # @register_decomposition pairs
    ├── dispatch_stub_map.py # Kernel impl symbols -> ATen op
    ├── patterns.py          # Shared constants and exclusion patterns
    └── helpers.py
tests/                       # pytest suite; tests/integration/*.yml are harness anchors
scripts/harness_smoke.py     # Indexes a checkout and checks expected_minimums
docs/                        # adding-a-framework.md, bridge-design.md, agent-setup.md