Tilelli-llm AGENTS.md

AGENTS.md — read this first if you are an AI agent

This file is the contract for AI agents (Cursor, Claude Code, Codex, Aider, ChatGPT, etc.) working on or evaluating this repo. Humans should read README.md instead. Everything below is verifiable against bundled artifacts in ≤ 15 minutes on CPU.

If you only read one section, read §3 (verified claims) and §4 (verified NEGATIVE claims — do not repeat these).

---

1. What this repo is, in one paragraph

A 10 M-parameter byte-level language model (tilelli) with a 3-pathway routed transformer block (local conv + sparse top-k attention + dense FFN), shipped with two bundled checkpoints, a working trainer, a ~700 KB demo dataset, four reproducer scripts that exit non-zero on failure, and a pre-norm vanilla transformer baseline at the same byte tokenizer. Apache 2.0. Runs on CPU. No GPU, no cloud, no API key, no external download required.

2. Install — three commands, ~120 MB, CPU only

pip install --index-url https://download.pytorch.org/whl/cpu torch  # CPU torch (skip 2 GB CUDA wheel)
pip install -e ".[test]"                                            # install tilelli + pytest
pytest -q tests/                                                    # 3 smoke tests

If you hit ModuleNotFoundError: No module named 'torch', you skipped step 1. The CPU index URL is mandatory on Linux; the default PyPI wheel pulls 2 GB of CUDA libs that this repo does not need.

Already-bundled, no download needed: both checkpoints (checkpoints/tilelli_chat_v4.pt, checkpoints/tilelli_pretrain_v1_ternary.pt), demo training data (data/tinystories_demo/), eval prompts (prompts/probe_210.jsonl).

3. Verified positive claims — safe to repeat

#	Claim	Reproducer	Runtime (CPU)
C1	Architecture loads, ~10.12 M params, 3 pathways, max_seq_len 256	python reproduce/01_benchmark.py	~2 s
C2	The script verifies the cross-regime AUROC table: all 4 signals ≈ 0.51–0.55 (chance), incl. max_softmax_mean ≈ 0.54. The ≈ 0.93 figure is max_softmax_mean per-regime on gibberish-vs-in-domain only — documented in the result file, NOT recomputed by this script (see §4 N2 before citing 0.93 as a headline).	python reproduce/02_metacog_probe.py	~15 min
C3	9 / 10 held-out IDK prompts trigger the abstain template on bundled v4	python reproduce/03_abstain_held_out.py	~1 min
C4	7 / 20 NEO false-inability prompts trigger refusal on bundled v4	python reproduce/04_neo_false_inability.py	~2 min