Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add latestaiagents/agent-skills --skill golden-set-maintenancegit clone --depth 1 https://github.com/latestaiagents/agent-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/latestaiagents/agent-skills/golden-set-maintenance)<a href="https://agentmods.dev/skills/latestaiagents/agent-skills/golden-set-maintenance"><img src="https://agentmods.dev/badge/skills/latestaiagents/agent-skills/golden-set-maintenance/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/latestaiagents/agent-skills/golden-set-maintenance"><img src="https://agentmods.dev/badge/skills/latestaiagents/agent-skills/golden-set-maintenance.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00107 | $0.01428 |
| Opus 5 | $0.00053 | $0.00714 |
| Sonnet 5 | $0.00021 | $0.00286 |
| Haiku 4.5 | $0.00011 | $0.00143 |
Grade A, and why
golden-set-maintenance scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 153 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Golden Set Maintenance
A golden set is 20-50 cases that matter most. If any fail, something important broke. Run them on every PR.
When to Use
- Your full eval takes > 10 min — too slow for per-PR
- You want a fast "does anything critical broken?" smoke check
- You need a stable reference for "what this system must always do"
- Safety-critical outputs where even one regression matters
What Goes In
High-signal cases ONLY. Each golden item should satisfy:
- Represents a core use case — if this fails, real users notice
- Unambiguous expected output — label is crisp, not subjective
- Has regressed at least once — historical anchor ("never again")
- Discriminating — different prompts/models yield different results
Reject:
- "Nice to have" improvements
- Flaky cases whose answer depends on time/state
- Items the model gets right 100% of the time across all candidates (not discriminating)
- Items no real user would actually submit
Size
- Smoke golden: 20-30 items, must run in < 2 minutes
- Core golden: 50-100 items, must run in < 10 minutes
Past 100, you're not golden anymore — you're a regression set (see regression-evals).
Curation Workflow
- Propose: anyone can add an item via PR. Include rationale ("this regressed in Oct 2025")
- Review: 2 reviewers verify the expected output is crisp and correct
- Label stability: the label shouldn't need updates as the product evolves
- Pass check: at least one model/prompt configuration should fail this case (otherwise not discriminating)
Selection Criteria Rubric
Before adding a golden item, answer:
- [ ] Is this a workflow real users actually do? (If no, don't add)
- [ ] Is the expected output objectively checkable? (If no, don't add)
- [ ] Would a 10% regression on this item be a P0 bug? (If no, don't add)
- [ ] Is there a similar item already in the set? (If yes, don't duplicate)
- [ ] Has a variation of this case failed before? (Bonus — strongly include)
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago First seen · 153 lines · 107 tokens per session scan A d1b8e9ab77ea
golden-set-maintenance is a skill published in the GitHub repository latestaiagents/agent-skills (5 stars, last pushed 4mo ago), licensed MIT. It adds 107 tokens to every session and 1,428 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
eval-driven-dev
Build an automated evaluation pipeline that tests a Python LLM application end-to-end with pixie test — real code paths, real LLM calls, instrumented external data — and scores outputs with evaluators instead of assertions. Use when adding evals to a Python AI app.
ai-enterprise-ai-usecase-priority-assessment
A business-diagnosis helper for deciding which enterprise AI use cases deserve attention first. It turns supplied information into a summary, findings, recommended actions, and reusable deliverables.
agent-benchmark-suite
Agent skill for benchmark-suite - invoke with $agent-benchmark-suite.
nemo-automodel-recipe-development
Create and modify NeMo AutoModel training and evaluation recipes, including YAML structure, builders, and execution flow.
neuron-test-engineer
Write tests for Neuron AI agents, RAG systems, workflows, and tools using the built-in testing utilities. Use this skill when the user mentions testing agents, writing unit tests, mocking AI providers, testing tool execution, verifying RAG retrieval, testing workflow behavior, or creating test cases for Neuron AI…
mem0-integration
Mem0 memory layer integration for AI agents. Implement persistent, semantic memory for long-term context retention and personalization.