ledger

ledger

!checklist init ${CLAUDE_SKILL_DIR} --force

A ledger is a record you commit entries to in order — and cannot quietly rewrite after the fact. ledger is the lens you hold over experiment design — research step three, after crucible has produced a surviving method — to turn that method into a protocol that produces evidence a reviewer believes. It is the third skill of the inquiry suite. Its product is one artifact: an experiment-protocol document, written and frozen before the main run, that doubles as the paper's experiments chapter and the reproduction package's README. It audits (and guides you to write) that protocol across gated stages, and it will not advance past a GATE until the checklist tool clears it. That gate enforces order — each step done before the next — not the substance of the work inside it; the tool structures the discipline, it does not audit it, so the rigor is yours to supply.

The one mental shift everything hangs on — write down what counts as evidence before you look at the data. The protocol — the claims, the instances, the baselines, the metrics, and the conditions that count as support or refutation — is committed before the main run. The reason is that the opposite order is p-hacking: run a pile of experiments, then notice a pattern, then build the story around it. You saw the pattern only after looking at many results, so it has a real chance of being noise you fished out — and a reviewer's nose for this is extremely good. Writing the verdicts first means you cannot move the goalposts after the ball is kicked.

The keystone — a firewall between exploration and confirmation. Chaos-testing a quick MVP — randomly kicking the system to see what breaks — is the fastest way to find what's real, and you should do it. But its output and the paper's evidence are two different things. So the work splits in two, and their data never mix. Exploration generates hypotheses ("component X seems to drag on large instances") into an append-only notebook; its numbers never enter the paper. Confirmation takes the believed hypotheses, writes them as claims with pre-written verdicts, and re-runs them on fresh seeds and fresh instances under the frozen protocol. Exploration discovers what might be true; confirmation proves what is. (If you already keep an append-only lab notebook, you have the exploration half right — this skill adds the confirmation half and the firewall between them.)

The agent is the experiment operator, never the oracle. Here the agent batch-submits jobs, watches them, auto-retries failures with the reason logged, and summarizes results into tables — the experiment operator's labor, run in parallel. But it is not the judge of what the numbers mean, and it optimizes for a good-looking result: it will not flag the data leak, and it will report the suspiciously-good number as a triumph. So your role shrinks to two things it cannot do — audit the protocol, and investigate anomalies: a number that is too good triggers a feasibility-and-leakage check first, never a celebration.

What you cannot delegate — the bets. Three calls stay yours: the verdicts written before the data (what counts as support, partial, refutation), the firewall (keeping exploration's fished-out patterns out of the confirmation evidence), and whether an anomaly is a bug or a result. Outsource these and you have automated a confident, fluent, p-hacked evaluation that dies in review.

Speak the user's language. Most calls here are the researcher's — which metric is the headline, is this baseline comparison fair, how many seeds is enough, is this finding real or fished. Read their field fluency and gloss a term on first use (the firewall, run provenance, a paired test, multiple-comparison correction, effect size, a performance profile, data leakage). A verdict the user can't evaluate is an opinion imposed, not a judgment shared.