Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add latestaiagents/agent-skills --skill golden-set-maintenancegit clone --depth 1 https://github.com/latestaiagents/agent-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/latestaiagents/agent-skills/golden-set-maintenance)<a href="https://agentmods.dev/skills/latestaiagents/agent-skills/golden-set-maintenance"><img src="https://agentmods.dev/badge/skills/latestaiagents/agent-skills/golden-set-maintenance/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/latestaiagents/agent-skills/golden-set-maintenance"><img src="https://agentmods.dev/badge/skills/latestaiagents/agent-skills/golden-set-maintenance.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00107 | $0.01428 |
| Opus 5.5 | $0.00043 | $0.00571 |
| Sonnet 5.5 | $0.00021 | $0.00286 |
| Haiku 4.5 | $0.00011 | $0.00143 |
Grade A, and why
golden-set-maintenance scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 153 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Golden Set Maintenance
A golden set is 20-50 cases that matter most. If any fail, something important broke. Run them on every PR.
When to Use
- Your full eval takes > 10 min — too slow for per-PR
- You want a fast "does anything critical broken?" smoke check
- You need a stable reference for "what this system must always do"
- Safety-critical outputs where even one regression matters
What Goes In
High-signal cases ONLY. Each golden item should satisfy:
- Represents a core use case — if this fails, real users notice
- Unambiguous expected output — label is crisp, not subjective
- Has regressed at least once — historical anchor ("never again")
- Discriminating — different prompts/models yield different results
Reject:
- "Nice to have" improvements
- Flaky cases whose answer depends on time/state
- Items the model gets right 100% of the time across all candidates (not discriminating)
- Items no real user would actually submit
Size
- Smoke golden: 20-30 items, must run in < 2 minutes
- Core golden: 50-100 items, must run in < 10 minutes
Past 100, you're not golden anymore — you're a regression set (see regression-evals).
Curation Workflow
- Propose: anyone can add an item via PR. Include rationale ("this regressed in Oct 2025")
- Review: 2 reviewers verify the expected output is crisp and correct
- Label stability: the label shouldn't need updates as the product evolves
- Pass check: at least one model/prompt configuration should fail this case (otherwise not discriminating)
Selection Criteria Rubric
Before adding a golden item, answer:
- [ ] Is this a workflow real users actually do? (If no, don't add)
- [ ] Is the expected output objectively checkable? (If no, don't add)
- [ ] Would a 10% regression on this item be a P0 bug? (If no, don't add)
- [ ] Is there a similar item already in the set? (If yes, don't duplicate)
- [ ] Has a variation of this case failed before? (Bonus — strongly include)
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 153 lines · 107 tokens per session scan A d1b8e9ab77ea
golden-set-maintenance is a skill published in the GitHub repository latestaiagents/agent-skills (6 stars, last pushed 6mo ago), licensed MIT. It adds 107 tokens to every session and 1,428 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
nexent-daily-test
A procedure for running Nexent daily, full-layer tests on a self-hosted Ubuntu machine. Nexent is tested across separate layers such as unit, service, AI runtime, and browser journeys, with results and evidence kept locally.
tfp-verification
Generate a deterministic report for disposable executor acceptance tests.
eval-driven-dev
Build an automated evaluation pipeline that tests a Python LLM application end-to-end with pixie test — real code paths, real LLM calls, instrumented external data — and scores outputs with evaluators instead of assertions. Use when adding evals to a Python AI app.
ai-enterprise-ai-usecase-priority-assessment
A business-diagnosis helper for deciding which enterprise AI use cases deserve attention first. It turns supplied information into a summary, findings, recommended actions, and reusable deliverables.
nemo-automodel-recipe-development
Create and modify NeMo AutoModel training and evaluation recipes, including YAML structure, builders, and execution flow.
rag-hybrid-search
Hybrid search combining semantic and keyword retrieval for RAG pipelines. Implement BM25 + dense vector search with fusion strategies.