weighted-compact CLAUDE.md

CLAUDE.md — weighted-compact root

Navigator for this repo. If you are Claude (or any LLM assistant) reading this checkout, read this file first and then jump to the relevant subdir. Do not read every file — use the map below.

Repo version: v0.3.0a1 (alpha; baseline run 2026-05-21) Status: alpha — the prior beta label was premature. Architectural invariants locked; numbers tunable. The one consumer with a positive, re-checkable result is recap.py (see README); the importance mixture has no measured fidelity edge and is documented as such.

---

Substrate framing

weighted-compact is not a compression tool. It is a substrate for self-distillation from your own Claude Code sessions.

The user's ~/.claude/projects/ already contains a record of every correction they pushed back on, every number they had to restate, every constraint the model lost track of. weighted-compact reads those files, runs them through a pipeline of measurable modules (extract_pairs → feature_extract → importance mixture → reconstruction-QA), and returns a compact memory shaped by the user's vocabulary, corrections, and reasoning.

Each module is a defined black box: known input, known output, documented in its own file. The boxes are replaceable independently. The quality metric driving development is reconstruction fidelity (can the compacted context answer questions about what was hidden from it?), not compression ratio.

If you are an LLM reading this repo to answer a question or make a change:

• The framing is "substrate" and "distillation", not "compressor" or "summarizer"
• The pipeline is a sequence of black boxes; treat them as independent
• The locked invariants (vectors-first, CAPTCHA labeling, no-harness-dep) are in the section below — do not propose changes that violate them

---

TL;DR in 30 seconds

• What this is: a trainable substrate that compacts Claude Code conversation history using vectors first, classifier as a refinement layer, with a CAPTCHA-style labeler for human-in-the-loop tuning.
• Architecture: three independent layers — substrate (extract_pairs + feature_extract over ~/.claude/projects/) → importance mixture (six signals composed continuously, plus a topic-decay multiplier on top) → reconstruction-QA (compression-fidelity gate, default local gemma3 judge with Sonnet 4.6 ground-truth calibration runs reported in docs/05-roadmap.md). Each layer disables cleanly if its dependencies are missing.
• Where it runs: localhost only, FastAPI on :18890, no external services.
• Privacy: substrate carries raw conversation text. Stays under $XDG_DATA_HOME/weighted-compact/, gitignored, never uploaded.