agent-prompt-audit

Agent Prompt Audit

An Agent's behavior comes from every piece of text that reaches it over its lifecycle, not just its system prompt. The audit's job is to find which text causes the behavior the user sees, on the backend and model that actually ran it, and to propose the smallest change that fixes it. Judge each instruction by what it does to behavior, not by its length: sometimes the fix is adding a missing reason or exit, and a clean surface is a valid result.

Deliver a report of findings — each with its evidence, confidence, and a concrete proposed change — and apply changes only when asked.

Core ideas

Evidence over reading. What Agents actually did beats what the text seems to say. Start from real runs and user corrections ("you stopped", "why didn't you report", "don't ask me that"), trace each symptom to the line that caused it or the missing line that would have prevented it, and use git blame to learn what incident a rule was written for and whether it still happens. A finding without evidence or documented model behavior is a flag, not a fix.

Context is kept; constraints must earn their place. Facts only the author knows — environment, contracts, ownership, quality bar, the reason behind a rule — are what prompts are for. Behavioral constraints are what go stale. Keep exact scripts where one sequence is safe (destructive commands, auth, merge gates, key custody), prohibitions against failures that still reproduce, and the scope bounds that make autonomy safe.

Say intent and reason, not pressure or method. Caps, MUST/NEVER, and emphasis without a reason make current models rigid; step scripts for judgment work and strategy coaching usually do worse than the model's own plan; fixed formats, word caps, and "don't narrate" rules produce silence or starved answers. Fossils — named-model workarounds, incident numbers as authority, "now/no longer" phrasing, one session's stumble made permanent — should become the current rule they stand for.