nanochat-rs-ternary: Instructions file for Claude Code

CLAUDE.md — nanochat-rs Ternary + mHC-lite Implementation Plan

For Claude Code. Execute phases in order. Each phase has a test gate — do not proceed to the next phase until the gate passes. All code lives in a Cargo workspace rooted at nanochat-rs-ternary/. Hardware target: dual AMD EPYC 9654 (224T, 1TB DDR5) + NVIDIA RTX PRO 6000 Blackwell (96GB).

Repository Layout (create first)

nanochat-rs-ternary/
├── Cargo.toml                    # workspace root
├── CLAUDE.md                     # this file
├── crates/
│   ├── ternary-core/             # Phase 1: packing, planar SoA, GGUF
│   │   ├── Cargo.toml
│   │   └── src/
│   │       ├── lib.rs
│   │       ├── encode.rs         # BitNet encoding (11=-1)
│   │       ├── pack.rs           # pack/unpack, group quantization
│   │       ├── planar.rs         # PlanarWeights: SoA layout, aligned alloc
│   │       ├── gguf.rs           # GGUF reader/writer for ternary types
│   │       └── verify.rs         # Triangle of Truth (Rust side)
│   │
│   ├── ternary-kernels/          # Phase 2: CPU + GPU compute kernels
│   │   ├── Cargo.toml
│   │   ├── build.rs              # cc::Build for C kernels, cuda compilation
│   │   ├── csrc/
│   │   │   ├── ternary_gemv.c    # adapted from ternary_final.c (v3.3.1)
│   │   │   ├── ternary_gemv.h    # C API header
│   │   │   └── ternary_dp4a.cu   # GPU decode kernel
│   │   └── src/
│   │       ├── lib.rs
│   │       ├── cpu.rs            # safe Rust wrappers over C FFI
│   │       ├── gpu.rs            # CUDA kernel launch wrappers
│   │       └── dispatch.rs       # runtime CPU feature detection + dispatch
│   │
│   ├── mhc-lite/                 # Phase 3: mHC-lite residual connections
│   │   ├── Cargo.toml
│   │   └── src/
│   │       ├── lib.rs
│   │       ├── n2.rs             # MhcLiteN2 (1 param, identity ↔ swap)
│   │       ├── n4.rs             # MhcLiteN4 (24 perms, full BvN)
│   │       ├── verify.rs         # doubly stochastic checks, composite gain
│   │       └── io.rs             # binary serialization (matches Python export)
│   │
│   ├── nanochat-model/           # Phase 4: transformer architecture
│   │   ├── Cargo.toml
│   │   └── src/
│   │       ├── lib.rs
│   │       ├── config.rs         # model configs (d20, 7B, 25B-MoE, 80B-MoE)
│   │       ├── embed.rs          # token + position embeddings
│   │       ├── norm.rs           # RMSNorm
│   │       ├── attention.rs      # MHA / GQA / MLA (with DeltaNet option)
│   │       ├── ffn.rs            # SwiGLU FFN (with MoE option)
│   │       ├── bitlinear.rs      # BitLinear: ternary GEMV dispatch
│   │       ├── block.rs          # TransformerBlock with mHC wiring
│   │       └── model.rs          # full model: embed → blocks → head
│   │
│   └── nanochat-serve/           # Phase 6: inference server
│       ├── Cargo.toml
│       └── src/
│           ├── main.rs           # CLI + Axum HTTP server
│           ├── engine.rs         # KV-cache, sampling, batched decode
│           └── api.rs            # OpenAI-compatible /v1/chat/completions
│
├── training/                     # Phase 5: PyTorch training
│   ├── mhc_lite.py              # mHC-lite module (BvN, exact DS)
│   ├── ternary_qat.py           # BitLinear STE, absmean quantization
│   ├── model.py                 # nanochat architecture in PyTorch
│   ├── train.py                 # training loop (Muon+Lion, WSD schedule)
│   ├── export.py                # PyTorch → GGUF + mHC binary export
│   └── requirements.txt
│
├── tests/                        # Integration tests
│   ├── triangle_of_truth.rs     # cross-validate all kernel paths
│   ├── mhc_property_tests.rs    # doubly stochastic invariants
│   ├── roundtrip_test.rs        # pack → GGUF → load → GEMV → verify
│   └── e2e_generate.rs          # full model forward pass sanity
│
└── benches/
    ├── gemv_bench.rs            # criterion benchmarks for all kernel paths
    └── mhc_overhead.rs          # verify mHC adds <0.001% overhead