postmortem

/postmortem

Analyze why an agent (LLM or pipeline) produced a flawed output. The goal is not to explain the technical mechanism but to identify the reasoning failure that led to the flaw being produced and not caught. The argument is a file path or description of the failure. Read relevant files first.

When to use this vs alternatives

• Use /postmortem when an agent or pipeline produced a flawed output that was presented as correct — the question is "why wasn't this caught?" not "what went wrong in the data?"
• Use /diagnose when the focus is on understanding empirical results — error distributions, root-cause hypotheses, validity. Diagnose analyzes data; postmortem analyzes agent reasoning.
• Use /critique for a broad review of an artifact's quality. Critique is proactive ("find problems"); postmortem is reactive ("explain this specific failure").

Key distinction

The question is never "why is the output wrong?" — it is "why was this output presented as correct?"

A wrong prediction is expected. A wrong prediction reported as a finding, deployed to a dashboard, or used to make a decision is a reasoning failure worth analyzing.

Procedure

1. Identify the flaw

State precisely what was wrong, with evidence. Quote the flawed output.

2. Trace the production chain

Walk backward through the chain that produced the flaw:

• Who generated it? (which agent, script, or pipeline step)
• What inputs did they have? (were the inputs sufficient to detect the flaw?)
• What checks were skipped? (was there a review step that should have caught it?)
• What structural condition enabled it? (missing verification gate, context window limits, anchoring on prior output, pattern-matching without verification, ungrounded generation beyond reliable recall)

3. Classify the failure mode

Which of these caused the flaw?

• Design-as-discovery: A constraint of the experimental setup was reported as an empirical finding. The agent failed to ask "could this have been different?"
• Layer misattribution: A property of one CI layer was attributed to another (e.g., L4 evaluation constraint reported as L1 model behavior)
• Momentum override: The agent was executing a plan and did not pause to verify intermediate results. Production outpaced reflection.
• Anchoring: The agent saw a number or pattern and built a narrative around it without checking the generating process.
• Missing mental model: The agent lacked understanding of how the metric/system works and could not detect the flaw even in principle.
• Context loss: The relevant information existed but was not in the agent's working context when the decision was made (e.g., schema defined in a different file than the analysis).
• Social proof: The agent reported something because it looked like what a finding "should" look like, not because it was verified.
• Ungrounded generation: The model produced plausible but factually false content because autoregressive generation cannot distinguish retrieval from novel generation. This is the default behavior of foundation models, not a situational failure — any generation task that exceeds reliably grounded knowledge will produce hallucinated output.