Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add iradoweck/antigravity-awesome-skills --skill ai-prompt-regression-testinggit clone --depth 1 https://github.com/iradoweck/antigravity-awesome-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/iradoweck/antigravity-awesome-skills/ai-prompt-regression-testing)<a href="https://agentmods.dev/skills/iradoweck/antigravity-awesome-skills/ai-prompt-regression-testing"><img src="https://agentmods.dev/badge/skills/iradoweck/antigravity-awesome-skills/ai-prompt-regression-testing/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/iradoweck/antigravity-awesome-skills/ai-prompt-regression-testing"><img src="https://agentmods.dev/badge/skills/iradoweck/antigravity-awesome-skills/ai-prompt-regression-testing.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00037 | $0.01371 |
| Opus 5.5 | $0.00015 | $0.00548 |
| Sonnet 5.5 | $0.00007 | $0.00274 |
| Haiku 4.5 | $0.00004 | $0.00137 |
Grade A, and why
ai-prompt-regression-testing scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
This is a copy
100% identical to ai-prompt-regression-testing — 0 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 119 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AI Prompt Regression Testing
When to Use
- Compare a prompt change with an observed baseline on a fixed evaluation set.
- Detect schema, factuality, refusal, tool-selection or task-completion regressions.
- Record a reviewable decision when results are noisy, missing or contradictory.
Workflow
- Define the task, representative cases and forbidden outcomes. Use synthetic or approved/redacted data; freeze case IDs and the dataset revision before runs.
- Record baseline/candidate prompt digests, exact available model/version, inference settings, tool versions and evaluation environment. Do not hardcode old model-family options or silently change models between comparisons.
- Predeclare checks and acceptance rules. Use deterministic schema/field checks for structured output, behavioral checks for tools and a clear human/rubric standard for open text. Similarity does not prove factual or code correctness.
- Run matched baseline/candidate cases under the same permissions and budget. Record latency/cost and repeated runs when variability matters. A timeout, missing output or judge error is incomplete, not a pass or a zero-cost result.
- Treat judge outputs as evidence with limitations. Blind/randomize presentation where feasible, calibrate against human-labeled examples, and record judge identity/rubric. Do not ask a model to verify private reasoning or accept a judge's confidence as an objective correctness measurement.
- Review differences case by case and report the denominators, failures and uncertainty. Keep mandatory safety/schema failures visible even if aggregate similarity improves. Use the predeclared rule, not a threshold selected after results.
- Store a redacted report and exact artifacts at an authorized destination. Decide retain/revise/reject for the stated dataset; deployment is separate.
Regression register
| Field | Type | Example / meaning |
|---|---|---|
| case_id | string | PTEST-001; a textual ID, not SERIAL |
| dataset_revision | immutable reference | exact case inventory, expected checks and labels |
| prompt_revision | string | baseline/candidate digest |
| model_runtime | string | exact available ID/version plus settings |
| check | string | required JSON fields or approved rubric criterion |
| result | enum | pass / fail / incomplete |
| evidence | reference | redacted actual output, tool trace or observed check |
| judge_configuration | optional object | exact judge ID/rubric/version and calibration limits |
| latency_ms | nonnegative number or null | null if not observed; never invented |
| cost | amount/currency or null | actual billing basis; unknown remains unknown |
| run_revision | immutable reference | binds the report to the actual run |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today Changed · -12 lines · +9 tokens per session 178bf2b9ac6c
- 3d ago First seen · 131 lines · 28 tokens per session scan A bd48f8169228
ai-prompt-regression-testing is a skill published in the GitHub repository iradoweck/antigravity-awesome-skills (30 stars, last pushed yesterday), licensed MIT. It adds 37 tokens to every session and 1,371 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. It is 100% identical to ai-prompt-regression-testing, differing in 0 lines, and is treated as a copy.
Other skills, from other repositories
ai-prompt-regression-testing
Use when comparing prompt changes against a fixed evaluation set: record actual model settings, required checks, missing results, judge limitations and case-level regression evidence.
ai-prompt-regression-testing
Prompt engineering regression test matrix register: baseline outputs, semantic drift thresholds, judge evaluations, and golden dataset.
ai-engineering-toolkit
Overview of six AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.
llm-engineering-expert
Build reliable applications on large language models: prompt design, structured output, evaluation, guardrails, and cost and latency control. Use when the user mentions LLMs, prompts, prompt engineering, few-shot examples, structured or JSON output, function calling, hallucination, model evaluation, token costs…
ai-engineering-toolkit
AI Engineering Toolkit workflow skill. Use this skill when the user needs 6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching and the operator…
llm-evaluation
LLM evaluation — automated metrics, human feedback, benchmarking. Use when testing performance, measuring AI quality, or establishing evaluation frameworks.