Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/microsoft/hve-core/evaluation-designnpx skills add microsoft/hve-core --skill evaluation-designgit clone --depth 1 https://github.com/microsoft/hve-coreWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00058 | $0.01250 |
| Opus 5 | $0.00029 | $0.00625 |
| Sonnet 5 | $0.00012 | $0.00250 |
| Haiku 4.5 | $0.00006 | $0.00125 |
Grade A, and why
evaluation-design scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 88 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AI Evaluation Dataset Design
Goal
Produce an evaluation dataset and its supporting documentation that measure whether an AI system does its job, refuses what it should refuse, and behaves acceptably under pressure. The dataset is a durable customer artifact, so its scope, balance, and rationale are recorded rather than implied.
Flow
- Run the scoping interview from the interview reference. Ask one question at a time and wait for the answer; do not batch the interview into a single prompt.
- Present a structured summary of what you heard and obtain explicit confirmation before generating anything.
- Derive the difficulty distribution from the confirmed scope, adjusting the defaults when the system's risk profile warrants it.
- Generate the dataset against the contract template in both machine-readable forms.
- Walk a representative sample through the user, gather consolidated feedback, and revise before finalizing the full set.
- Produce one sectioned evaluation guide containing curation notes, metric selection with rationale, and tooling recommendations.
- Route every durable write through the workstream's scan gate before it lands in a customer location.
Inputs
- The system under evaluation, its purpose, and its intended users
- Its grounding sources, tools, and response-format expectations
- Known risks, refusal requirements, and prohibited content areas
- The team's development approach and evaluation cadence
- A caller-confirmed destination for the dataset and documents
Success criteria
- Every interview area is answered or explicitly recorded as unknown before generation begins.
- The dataset meets its size floor and its confirmed category balance, and the recorded difficulty counts match the actual rows.
- Every pair records the confirmed user populations it exercises, and every confirmed population carries a count, including the ones with no pairs.
- Each pair states its category, difficulty, expected behavior, and, where relevant, the tools the system should invoke.
- Refusal and safety pairs assert the specific action expected, not merely that the system declines.
- The sample review happened and its feedback is reflected in the final set.
- Metric selection is justified from the system's actual grounding, tool use, and risk profile rather than applied uniformly.
- The evaluation guide states who reviewed the content and when it should be revisited.
What ships with it
5 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 88 lines · 58 tokens per session scan A 7dcef9291bc4
evaluation-design is a skill published in the GitHub repository microsoft/hve-core (1,411 stars, last pushed today), licensed MIT. It adds 58 tokens to every session and 1,250 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
systematic-debugging
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes.
brainstorming
You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. Explores user intent, requirements and design before implementation.
chat-pet-sprite-creation
Use when creating or changing VS Code chat pet sprite art, sprite sheets, state animations, eye treatments, Stable/Insiders variants, or pet transitions under src/vs/workbench/contrib/chat/browser/widget/media/chatPet.
cpu-profile-analysis
Analyze V8/Chrome CPU profiles (.cpuprofile) and DevTools trace files (Trace-.json). Use when: profiling performance, investigating slow functions, comparing code paths, finding bottlenecks, analyzing timeToRequest, understanding call trees from sampling profiler data, analyzing layout/paint/rendering, investigating…
agent-host-chat-contributions
Build and review cross-cutting agent-host chat behavior through lifecycle contributions. Use when adding turn lifecycle side effects, prompt or context injection, restored-history transformation, protocol-action observation, or when reviewing changes that add code to AgentSideEffects or AgentService.
auto-perf-optimize
Run agent-driven VS Code performance or memory investigations. Use when asked to launch Code OSS, automate a VS Code scenario, run the Chat memory smoke runner, capture renderer heap snapshots, take workflow screenshots, compare run summaries, or drive a repeatable scenario before heap-snapshot analysis.