vally-tests

Vally Tests Skill

Purpose

This skill authors Vally conformance tests for the four supported artifact kinds in this repository: prompts, instructions, agents, and skills. Each test exercises a documented behavior the artifact already claims and routes the result through an appropriate Vally grader so failures are explainable. Test authoring is bounded by a refusal taxonomy that keeps the skill out of adversarial, harmful, or policy-evasion territory.

Follow the shared content-policy public-output guard for eval stimuli and any public summaries derived from this skill. Vally tests are not a venue for terms-of-service boundary mapping, payload generation, refusal-text scoring, hidden-instruction disclosure, PII extraction, or secret extraction.

The skill ships:

• A canonical authoring workflow used by the Vally prompts and Vally Test Author subagent.
• Per-kind reference files that enumerate every conformance check the skill knows how to express.
• A grader catalog that maps Vally CLI 0.9.0 grader types to the checks they fit.
• A safety refusal taxonomy with regex patterns the safety lint script consumes.
• Helper scripts and asset templates for stimulus emission, corpus import, and dedupe.

When to Invoke

Invoke this skill in one of two modes:

• From-artifact mode. The caller points at one artifact file (a .prompt.md, .instructions.md, .agent.md, or SKILL.md) and asks for conformance test stimuli that verify the artifact's stated behaviors. The skill detects the artifact kind from the filename, looks up the matching per-kind reference, picks graders, and appends stimulus blocks to the routed Vally eval file.
• Corpus-import mode. The caller supplies a CSV or XLSX corpus that follows the shipped template, and the skill turns each row into a stimulus block in the routed eval file. Used when a large batch of behaviors needs identical formatting and dedupe handling.

Do not invoke this skill to:

• Generate adversarial probes, jailbreak attempts, prompt-injection payloads, or red-team stimuli.
• Test refusal quality on harmful, illegal, or policy-violating prompts.
• Author stimuli that elicit PII, secrets, hidden instructions, or model self-disclosure.
• Store payload examples, paraphrased prohibited requests, or quoted flagged content in Vally eval specs, corpus rows, PR summaries, issue bodies, or comments.
• Replace Responsible AI threat modeling. RAI work lives in the RAI Planner agent.