replayd: Instructions file for Codex

replayd — Three Agent Examples

Turn failed AI agent runs into replayable regression tests.

This document covers the three example agents in the examples/ directory and explains how each one demonstrates the replayd release-control loop.

---

Table of Contents

1. What replayd does
2. The four-step loop
3. Agent integration contract
4. Example 1 — Multi-step Planning Agent
5. Example 2 — RAG Policy Agent
6. Example 3 — Production Incident Response Agent
7. Running everything
8. Test results
9. Grading reference

---

What replayd does

AI agents regress silently. A team fixes a bug, ships a new prompt or model, and the same bad behavior quietly returns. Traditional software has regression tests and CI/CD to catch this. AI agents have had nothing equivalent.

replayd is the fix. It captures a failed agent run in full — input, output, every tool call — marks it as a known failure, saves it as a regression test, and then replays that test against every future version of the agent before you ship.

capture → mark_failed → save_test → replay_all

If the same bad behavior returns, the release is blocked. That is the entire idea.

---

The four-step loop

from replayd import Replayd

rp = Replayd()

# 1. Capture a run
with rp.capture(input=user_input, model="gpt-4o") as run:
    run.output = your_agent(user_input, run)

# 2. Mark it as failed
rp.mark_failed(run.id, reason="agent skipped constraint check")

# 3. Save as a regression test
rp.save_test(
    run.id,
    forbidden_actions=["finalize_plan"],
    expected_action="check_constraints",
)

# 4. Replay before every future deployment
results = rp.replay_all(agent=your_agent)
for r in results:
    print(r.verdict, r.reason)