epistemic-protocols: Skill for Claude Code

Outcome Eval

Without a protocol, a person has to recall at length and write a long opening prompt. With one, they answer when asked, by recalling, or recognize what is put in front of them — and the items that come up this way can be ones nobody would have thought of at the start. This eval shows that one thing, from the transcript: the same request runs with and without the protocol, and each cell lists the decisions that reached the user through the AI.

The one measure

Per cell: the decision items that entered the conversation through the AI and that the user's opening request did not contain. Each is recorded with the verbatim span that raised it and how it reached the user:

• asked — put as a question; the user answered by recalling.
• presented — shown as an option or default; the user recognized, picked or corrected it.

The number is the count of such items, per cell and per arm, shown under the opening request whose gaps they fill: each item is something the user would otherwise have had to write into that request. It is read from the transcript and the opening request alone — no hidden specification, no checklist, no grader of correctness. Identifying an item is a reading of the transcript, attributed to whoever wrote the cell's notes, not a mechanical extraction; the spans stay with it so anyone can check the reading.

What this eval measures, and what it does not

It shows how many decisions reached the user as a question or as something to recognize, rather than having to be written up front, and how many of them reached the user before any code existed — the only ones an answer could shape before the implementation. It does not show whether an answer then changed the code, whether the answers were right, whether a person felt less load, or anything about rework.

A result is an observation of one model on one day, and it changes as models change. A write-up quotes every reportable cell of the runs it draws on, including cells where the arms did not differ and cells where the bare arm raised more; picking cells turns an observation into a showcase. It goes into the pull request body or the commit message it informs, never onto a state surface — a README, an AGENTS.md, this file — where nothing re-runs it. A surface may say that this eval exists and what it counts; it does not say what it found. Records (results/) are gitignored for the same reason.