Getting it into your agent
It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.
git clone --depth 1 https://github.com/alivirgo/Major-AI-Skillsnpx agentmods add skills/alivirgo/major-ai-skills/prompt-regression-gateWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/alivirgo/major-ai-skills/prompt-regression-gate)<a href="https://agentmods.dev/skills/alivirgo/major-ai-skills/prompt-regression-gate"><img src="https://agentmods.dev/badge/skills/alivirgo/major-ai-skills/prompt-regression-gate/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/alivirgo/major-ai-skills/prompt-regression-gate"><img src="https://agentmods.dev/badge/skills/alivirgo/major-ai-skills/prompt-regression-gate.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00053 | $0.03731 |
| Opus 5.5 | $0.00021 | $0.01492 |
| Sonnet 5.5 | $0.00011 | $0.00746 |
| Haiku 4.5 | $0.00005 | $0.00373 |
Grade A, and why
prompt-regression-gate scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 21d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 357 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Prompt Regression Gate AI Skill Guide (Claude)
Overview & Engine Architecture
A prompt is production code. Unit tests do not catch “fluent but worse.” A regression gate answers only: is this change worse than the pinned baseline on the same frozen cases? Absolute “looks good” review and candidate self-confidence are not gates.
Claude operates as a Principal LLM Release Engineer, specializing in paired baseline vs candidate runs, relative delta thresholds, slice/P0 hard fails, stochastic control, and hash-bound provenance (prompt, dataset, system, judge).
┌─────────────────────────────────────────────────────────────┐
│ Prompt Regression Gate │
│ │
│ Inputs (frozen before compare) │
│ ├── dataset_version + dataset_hash (@ai-evaluation-dataset)│
│ ├── baseline: prompt_hash + model + tools + retriever │
│ ├── candidate: same knobs except the intentional delta │
│ └── thresholds agreed BEFORE seeing scores │
│ │
│ Runner │
│ ├── Paired per-case execution (identical context) │
│ ├── Deterministic scorers + optional pinned LLM judge │
│ └── Provider errors tracked separately from quality fails │
│ │
│ Gate │
│ ├── Aggregate delta vs baseline within tolerance │
│ ├── Slice / P0: no critical regressions │
│ └── Verdict: go | no-go | untested + rollback plan │
└─────────────────────────────────────────────────────────────┘
When to use / when not to
Use when
- A PR touches prompts, templates, tool/JSON schemas, packers, model IDs, or retrieval config that changes model context.
- Promoting a model upgrade or temperature/tool change.
- You need a merge-blocking CI check, not a dashboard vibe.
What ships with it
2 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 21d ago Changed · +322 lines · +24 tokens per session 3867e4c86393
- 1mo ago First seen · 35 lines · 29 tokens per session scan A b66dcbe3b575
prompt-regression-gate is a skill published in the GitHub repository alivirgo/Major-AI-Skills (2 stars, last pushed today), licensed MIT. It adds 53 tokens to every session and 3,731 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-12.
Other skills, from other repositories
llm-tester
You are the LLM Tester, specializing in systematic prompt evaluation, red-teaming, and LLM output quality assurance. You replace "vibes-based" AI evaluation with rigorous, automated, and repeatable verification suites.
advanced-evaluation
Design LLM-as-judge rubrics, direct scores and pairwise comparisons. Use for judge models, evals, position bias and calibration against human labels.
agent-harness-fault-injection
Use when an agent workflow needs deterministic recovery evidence for sandbox, MCP/tool, worker, checkpoint, memory, or orchestration failures.
ai-engineering-toolkit
Overview of six AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.
ai-prompt-regression-testing
Use when comparing prompt changes against a fixed evaluation set: record actual model settings, required checks, missing results, judge limitations and case-level regression evidence.
api-contract-testing
Design and review API contract tests across providers and consumers. Use for schema changes, compatibility regressions and client integration failures.