aidd-riteway-ai

🧪 aidd-riteway-ai

Act as a top-tier AI test engineer to write correct riteway ai prompt evals for multi-step agent skills that involve tool calls.

Refer to /aidd-tdd for assertion style (given/should/actual/expected) and test isolation principles.

Refer to /aidd-requirements for the "Given X, should Y" format when writing assertions inside .sudo eval files.

---

Process

1. Read the skill under test and its functional requirements
2. Identify the discrete steps in the skill's flow
3. Create one .sudo eval file per step (Rule 1), placed in ai-evals/<skill-name>/
4. For each file, write the userPrompt — include mock tool preambles for unit evals (Rule 2), assert tool calls for step 1 (Rule 3), supply previous step output for step N > 1 (Rule 4)
5. Write assertions derived strictly from functional requirements in Given X, should Y format (Rule 7)
6. Create small, single-condition fixture files as needed (Rule 6)
7. Verify against the Eval Authoring Checklist below

---

Eval File Structure

A .sudo eval file has three sections:

import 'ai/skills/<skill-name>/SKILL.md'

userPrompt = """
<prompt sent to the agent under test>
"""

- Given <condition>, should <observable behavior>
- Given <condition>, should <observable behavior>

Assertions are bullet points written after the userPrompt block. Each assertion tests one distinct observable behavior derived from the functional requirements of the skill under test.

---

Rule 1 — One eval file per step

Given a multi-step flow under test, write one .sudo eval file per step rather than combining all steps into a single overloaded userPrompt.

Naming convention:

ai-evals/<skill-name>/step-1-<description>-test.sudo
ai-evals/<skill-name>/step-2-<description>-test.sudo

Do not collapse multiple steps into one file. Each file tests exactly one discrete agent action.

---

Rule 2 — Unit evals: tell the agent it is in a test environment

Given a unit eval for a step that involves tool calls (gh, GraphQL, REST API), include a preamble in the userPrompt that: