rainman CLAUDE.md

Rainman — Project Instructions

Read this before doing anything.

What This Project Is

Rainman is a standalone developer memory tool that plugs into AI coding workflows via MCP and Claude Code hooks. It remembers what you've built, what failed, what works — and surfaces relevant knowledge when the AI needs it, without being asked.

Zero LLM calls. Runs locally. Zero external dependencies (stdlib only). Storing and ranking memory spends zero tokens; recalled memories are injected as normal context, so they cost input tokens only when actually surfaced to the model.

Built by extracting the scoring engine from CogniTrait (Pygmalion's personality-shaped memory), stripping Big Five personality dependencies, and adapting it for project knowledge retrieval.

Repo: C:\Users\yanko\My Apps\rainman Stack: Python 3.10+ (stdlib only) Tests: pip install -e . && pytest tests/ -m unit — 350 tests, stdlib only. (The team sync server and its tests live in the separate rainman-server repo.)

Architecture

rainman/
  core/
    models.py       Memory (+ author, .trust property) + RecallResult dataclasses
    scoring.py      IDF-weighted (stemmed) keyword, temporal decay, importance, associative scoring + trust amplifier denial + quality prior
    text.py         Tokenization: stem + stopword filter + IDF index (shared by scorer)
    porter.py       Vendored public-domain Porter stemmer (stdlib, zero-dep)
    sentiment.py    Keyword-based sentiment classifier (zero LLM)
    trust.py        Trust levels (user>hook>ingest) derived from source; quality prior
    identity.py     current_actor() — local OS user / RAINMAN_AUTHOR
    engine.py       Core: add (w/ dedup+supersession), recall (task-conditioned), context, links, forget, persist, retention, _visible()
    store.py        JSON backend (default): layered persistence, file locking, fsync, schema_version
    sqlite_store.py SQLite backend (opt-in): WAL, per-layer DB, no 2000-cap, indexed
    storage.py      StorageBackend Protocol + make_store() backend factory
    redact.py       Secret redaction + path denylist (+ org-policy extras) for auto-learn safety
    salience.py     Write-side curation: score how worth-remembering an auto-learned memory is (gate hook spam)
    consolidate.py  Offline "sleep" pass: episodic->semantic generalization (extract common elements across recurring events) + functional forgetting (Ebbinghaus adaptive decay of un-recalled orphan noise). Zero-LLM. Wired via engine.consolidate() / `rainman consolidate`.
    gaps.py         Skill-gap report: cluster recurring failures, rank by frequency/recency/unresolved -> "which skill or permanent fix to write next" (`rainman gaps`)
    working.py      Working-memory buffer: capacity-limited (7±2) TTL'd LRU of the memories in focus this session (Miller/Baddeley). Persisted to .rainman/working.json; recall() touches it; engine.working_set() / `rainman working`.
    audit.py        Append-only JSONL audit log (opt-in, batched) — store/recall/forget/retention
    config.py       Policy control plane (org.enforce > project > user > org.defaults > builtin)
    log.py          Structured stdlib logging (RAINMAN_LOG_LEVEL)
    fusion.py       Reciprocal Rank Fusion (combines lexical + dense rankings)
  eval/
    metrics.py      IR metrics: recall@k, precision@k, MRR, nDCG@k (stdlib)
    harness.py      Gold-set retrieval eval -> EvalReport (the IR gate runner)
    agent_harness.py  memory-on vs memory-off lift plumbing (SWE-bench)
  mcp/
    server.py       MCP stdio server (JSON-RPC 2.0, 5 tools)
  cli/
    commands.py     CLI command implementations (init, add, recall, status, setup, doctor)
  integration/
    core.py         HOST-AGNOSTIC behaviours (auto_pull, session_start_context, compaction_context, learn_from_tool, capture_learnings, learn_from_commit). The hooks are thin adapters over this; other hosts (git, aider, MCP) reuse it. Zero host coupling.
    hosts.py        Registry of MCP-capable hosts (Cursor/VS Code/Windsurf/Cline/Zed/Continue/Claude) + config shapes; powers `mcp-config` / `setup --host`
  hooks/             (Claude Code ADAPTERS — parse Claude's stdin/transcript, delegate to integration/core)
    session_start.py   Load project context at session start (also handles post-compaction re-injection)
    post_compact.py    Legacy compaction hook (logging only; re-injection moved to session_start)
    post_tool_use.py   Auto-learn from file reads, edits, test runs (salience-gated; Bash outcomes -> typed-causal experience cards w/ failure->fix pairing)
    session_end.py     Capture key decisions from conversation transcripts
  sync/
    client.py       SyncClient — push/pull project memories to a sync server (stdlib urllib)
  semantic/
    __init__.py     OPTIONAL semantic lane SEAM: provider protocol + cosine + loader (None unless rainman[semantic] installed)
  ingest/
    git.py          Parse git log into memories
    files.py        Scan project file tree into memories
  __main__.py       CLI entry point (argparse)

The self-hosted TEAM SYNC SERVER now lives in a SEPARATE REPO:
  https://github.com/yan-yanko/rainman-server  (BSL 1.1, source-available)
It holds the server (RBAC, OIDC SSO, audit, encryption-at-rest, admin console)
plus its SOC2-readiness doc and the client<->server integration tests. This
repo (the client) stays MIT + stdlib-only. `rainman/sync/client.py` is the
client half that talks to it via `rainman remote` / `rainman sync`.
tests/                  (350 tests total, all marked `unit`)
  test_scoring.py     scoring components + weighted sum
  test_engine.py      add / recall / context / links / forget
  test_sentiment.py   sentiment classifier
  test_hooks.py       session_start, post_compact, post_tool_use, session_end
  test_mcp_server.py  MCP JSON-RPC protocol + tools (incl. error sanitization)
  test_cli_smoke.py   CLI smoke
  test_integration.py end-to-end layering
  test_concurrency.py locking, corruption quarantine, fsync, schema version
  test_regressions.py regression guards
  test_trust.py       trust levels, amplifier denial, quality prior, floors (Ph1a)
  test_audit.py       append-only audit log (Ph1b)
  test_config.py      policy precedence + wired knobs (Ph1c)
  test_retention.py   TTL prune + global-layer save safety (Ph1d)
  test_review.py      quarantine review queue: approve/reject (Ph2c)
  test_salience.py    write-side salience scoring + threshold (M6)
  test_experience.py  typed-causal cards: record/find/resolve failure->fix pairing (M1)
  test_consolidation.py  dedup/merge near-duplicates + supersession (M5)
  test_assoc_graph.py  real linking (stemmed/windowless/typed edges) + 2-hop spreading activation (M3/M4)
  test_semantic.py  optional semantic lane seam: RRF fusion + stub-provider synonym recall (M7)
  test_eval.py      IR metrics + gold-set harness + memory-lift on/off plumbing
  test_gaps.py      skill-gap report: failure clustering, ranking, known-fix surfacing
  test_sqlite_backend.py  SQLite backend parity, selection, migrate (Ph2a)
  test_sync_client.py client-side sync: config/token-safety, push/pull apply (mocked HTTP)
  test_retrieval_quality.py  IR gate: recall@5/MRR on paraphrased queries + relevance floor

(Server-side + client<->server integration tests live in the rainman-server repo.)