agent-guardrails

Poka-Yoke for AI-Written Code

An agent is a fast, tireless operator with no memory of yesterday and a strong prior toward appearing successful. That is the exact profile Shingo designed poka-yoke for, except an agent makes mistakes faster than any human, and never learns from the ones you correct in conversation.

The governing insight: instructions to an agent are rung zero. A line in CLAUDE.md saying "never commit to main" is training, and training degrades, under long contexts, compaction, and subagents that never read the file. A PreToolUse hook that denies the push is a device. If you have been repeating the same correction to an agent, that is the signal to stop writing instructions and install a device.

A complete answer covers all five

The diagnosis is not the answer. "Instructions are not enforcement" is the right insight, and it is satisfying to write, but someone asking "what am I doing wrong?" has a repo they need to fix: not a question about their prose. Explaining why the rules fail and stopping there leaves them exactly where they started. State the insight in a sentence, then spend the rest of the answer on the replacement.

Replacing an instruction with a device is not one step, it is five, and stopping after the first leaves the person with a rule that looks enforced and is not. Naming the deny rule is the easy part and the least of it. Cover every one of these, briefly, before adding depth:

1. The deny rule, with real syntax. Show the actual permissions.deny entry for their case, "Bash(git push --force:*)": not a description of one. A pattern they have to invent themselves is a step where this fails.
2. A hook where a pattern is not enough. Deny rules match strings. Anything conditional: a DELETE without a WHERE, an edit allowed in one directory but not another, a production hostname, needs a PreToolUse hook that inspects the call and returns a deny. Say which of their two rules needs which.
3. What the deny message says. The agent reads it and acts on it, so a bare refusal produces a workaround, often a worse one. The message must name what was blocked, why, and what to do instead. This is the one place prose belongs in a device.
4. Where the config lives, so it applies to everyone. .claude/settings.json, committed. A rule in settings.local.json protects one machine, which is the same failure as documenting it: the protection exists only where someone remembered to set it up.
5. Proof that it fires. Run the blocked action and confirm the denial and its message, then run the legitimate neighbouring action and confirm it still works. Untested hooks fail open more often than people expect: a regex that does not match the real command string is a hook that does nothing while looking like protection. An unverified device is worse than no device, because it creates confidence without protection.