Skills for Copilot Studio is a plugin for authoring, testing, and troubleshooting standard Microsoft Copilot Studio agents as YAML files from a terminal or editor. It is intended for users of Claude Code, GitHub Copilot CLI, and VS Code who work with Copilot Studio agents. The catalogue add-ons are the plugin's skills, agents, hook, and plugin definition.
Getting it into your agent
It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.
git clone --depth 1 https://github.com/microsoft/skills-for-copilot-studionpx agentmods add skills/microsoft/skills-for-copilot-studio/create-evalWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/microsoft/skills-for-copilot-studio/create-eval)<a href="https://agentmods.dev/skills/microsoft/skills-for-copilot-studio/create-eval"><img src="https://agentmods.dev/badge/skills/microsoft/skills-for-copilot-studio/create-eval.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00041 | $0.01796 |
| Opus 5 | $0.00020 | $0.00898 |
| Sonnet 5 | $0.00008 | $0.00359 |
| Haiku 4.5 | $0.00004 | $0.00180 |
Grade A, and why
create-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 174 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Create Scenario Eval
Guide the user through creating eval test cases for a Copilot Studio plugin scenario. Evals test end-to-end scenarios with natural prompts — the request routes through sub-agents (e.g., Author agent) which invoke skills internally.
How the eval system works
The eval harness (evals/evaluate.py) works by:
- Copying a fixture agent into a temp workspace
- Running
claude -p "<prompt>"with a PreToolUse hook that traces skill invocations inside sub-agents - Checking routing (which agents and skills were invoked), output files, and response text against deterministic checks
- Producing a JSON results file and HTML report
What can be tested right now
Authoring scenarios that produce YAML files (topics, agents, knowledge sources, etc.) are the best candidates. The harness supports these check types:
| Check | What it validates | Use for |
|---|---|---|
agent_invoked |
Expected sub-agent was dispatched (e.g., Author agent) | Routing verification |
agent_not_invoked |
Unwanted sub-agents were NOT dispatched | Routing verification |
skill_invoked |
Expected skill was invoked (traced inside sub-agents via hook) | Skill routing |
skill_not_invoked |
Unwanted skills were NOT invoked | Skill routing |
files_created |
Expected files were created/modified (glob pattern) | All authoring scenarios |
schema_validate |
Full Copilot Studio schema validation (kind, required fields, IDs, Power Fx, scopes) | All YAML-producing scenarios |
yaml_structure |
Specific YAML path has expected value, min array length, or contains string | Structural assertions |
content_contains |
Keywords from prompt appear in output files | Domain relevance |
no_placeholders |
No _REPLACE, TODO, or FIXME markers left |
Template completion |
stdout_contains |
CLI response text contains expected strings | Reference/info scenarios |
stdout_not_contains |
CLI response does NOT contain error strings | Error absence |
exit_code |
CLI exited with expected code | All scenarios |
yaml_unchanged |
Specific file or YAML path was NOT modified | Preservation testing |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago First seen · 174 lines · 41 tokens per session scan A d30b01a2205e
create-eval is a skill published in the GitHub repository microsoft/skills-for-copilot-studio (432 stars, last pushed 4d ago), licensed MIT. It adds 41 tokens to every session and 1,796 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
research-engineer
An uncompromising Academic Research Engineer. Operates with absolute scientific rigor, objective criticism, and zero flair. Focuses on theoretical correctness, formal verification, and optimal implementation across any required technology.
tika-eval-compare
Compare extracts from two Tika builds over a corpus to detect regressions in content, encoding, exceptions, and embedded-document handling. Use for "compare before/after extracts", "eval this change against the corpus".
neuron-evaluation-engineer
Create and run AI evaluations with datasets, assertions, and output drivers in Neuron AI. Use this skill whenever the user mentions evaluation, testing AI systems, creating evaluators, dataset-driven testing, assertion-based validation, or wants to measure AI system performance. Also trigger for tasks involving…
jetson-validate-image
Use after jetson-flash-image to run static BSP checks, on-target smoke/regression tests on a flashed DUT, or both. Not for build or flash steps. Triggers: validate bsp, on-target validation.
atmos-validation
Validate Atmos projects, components, arbitrary JSON Schema inputs, EditorConfig, and GitHub Actions; use affected-file selection and native CI annotations.
skill-benchmark
Benchmark AI skill effectiveness by measuring implementation quality against legacy constraints.