Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add ai-analyst-lab/ai-analyst --skill evalgit clone --depth 1 https://github.com/ai-analyst-lab/ai-analystWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/ai-analyst-lab/ai-analyst/eval)<a href="https://agentmods.dev/skills/ai-analyst-lab/ai-analyst/eval"><img src="https://agentmods.dev/badge/skills/ai-analyst-lab/ai-analyst/eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/ai-analyst-lab/ai-analyst/eval"><img src="https://agentmods.dev/badge/skills/ai-analyst-lab/ai-analyst/eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00045 | $0.00509 |
| Opus 5 | $0.00023 | $0.00254 |
| Sonnet 5 | $0.00009 | $0.00102 |
| Haiku 4.5 | $0.00005 | $0.00051 |
Grade A, and why
eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 41 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Evaluate the system
Before running
Name the exact system under test. Record its model, instructions, skills, agents, helpers, knowledge, workflow, tools, connector configuration, and data snapshot.
Use one of these modes honestly:
- Working mode supports iteration. Its references may be visible to the evaluator, but never to the child trial before its output is locked.
- Course heldout mode sends locked outputs to the course-controlled grader. The expected results do not live in the student clone.
- A local visible answer file is development material. Do not call it a secret heldout evaluation.
Run
- Load the question-only manifest from
data/evals/public/. - Select the split and any named slice before the run starts.
- Use
helpers.evals.controller.EvaluationControllerto launch and record the trials. - Give each trial only its public task, permitted system files, permitted data, and permitted tools.
- Lock every trial output before grading begins.
- Grade deterministic criteria first. Keep model-based grades separate.
- Preserve pass, fail, blocked, error, invalid, and unknown as different results.
- Report every case and slice before discussing the aggregate.
The local controller is available through python3 -m helpers.evals.cli run-suite. Use --model claude-opus-4-6. General code access is not required for routing or contract cases. When local data analysis requires --allow-code, state that local process isolation is not the same as course-heldout answer isolation.
For a reviewed working suite with local references, lock the trial outputs first, then grade them
with python3 -m helpers.evals.cli grade-suite. Pass the run ID, public manifest, and reviewed
reference file. Never copy the reference file into the trial workspace.
Compare a change
Hold the suite, data snapshot, model, evaluator, tools, and trial count fixed. Name one intended system change. If more than one material input changed, label the comparison confounded rather than attributing the score movement.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 41 lines · 45 tokens per session scan A c9fc9f395aab
eval is a skill published in the GitHub repository ai-analyst-lab/ai-analyst (298 stars, last pushed 3d ago), licensed MIT. It adds 45 tokens to every session and 509 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-12.
Other skills, from other repositories
systematic-debugging
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes.
local-ai-agents
Build local-first AI agents that run entirely on a developer workstation with Microsoft Foundry Local and Qwen function-calling models. Covers Small Language Models (SLMs), the OpenAI-compatible local endpoint, sandboxed local tools, local RAG with Chroma, local MCP servers, hybrid cloud/local routing, and the…
next-cache-components-adoption
Turn on Cache Components in a Next.js app and resolve the blocking routes it surfaces. Use when the user wants to enable, adopt, or migrate to Cache Components, flip the cacheComponents flag, work through a flood of blocking-prerender / instant validation errors, run the cache-components-instant-false codemod, or…
chronicle
Analyze Copilot session history for standup reports, usage tips, session search, and session reindexing. Use when the user asks for a standup, daily summary, usage tips, workflow recommendations, wants to search or find past sessions by keyword/file/PR, wants to reindex their session store, or asks about deleting…
chat-pet-sprite-creation
Use when creating or changing VS Code chat pet sprite art, sprite sheets, state animations, eye treatments, Stable/Insiders variants, or pet transitions under src/vs/workbench/contrib/chat/browser/widget/media/chatPet.
cpu-profile-analysis
Analyze V8/Chrome CPU profiles (.cpuprofile) and DevTools trace files (Trace-.json). Use when: profiling performance, investigating slow functions, comparing code paths, finding bottlenecks, analyzing timeToRequest, understanding call trees from sampling profiler data, analyzing layout/paint/rendering, investigating…