verify-samples-tool

A guide to verify-samples, a tool in the .NET Agent Framework repository that runs sample projects and checks whether their output is correct. It can use fixed checks and AI-based verification.

In plain words
What is it for?
Use it when adding, changing, building, or verifying .NET sample projects, including controlling parallel runs or building samples during verification.
Why use it?
It removes the need to run and inspect each sample manually. It also records results in log or CSV files and can run samples by category or name.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/microsoft/agent-framework/verify-samples-tool
Any agent
npx skills add microsoft/agent-framework --skill verify-samples-tool
Clone the repo
git clone --depth 1 https://github.com/microsoft/agent-framework

Made for: Claude Code, Codex.

Per session 40 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 2,123 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00040 $0.02123
Opus 5 $0.00020 $0.01061
Sonnet 5 $0.00008 $0.00425
Haiku 4.5 $0.00004 $0.00212

Measured yesterday against content hash dbec3e9727ff, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

verify-samples-tool scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

dotnet/.github/skills/verify-samples-tool/SKILL.md · 228 lines

How it starts

The opening of the file, as written. The whole thing — 228 lines — stays where its author put it; the contents beside it link to each section on GitHub.

verify-samples Tool

The verify-samples project (dotnet/eng/verify-samples/) is an automated tool that runs sample projects and verifies their output using deterministic checks and AI-powered verification.

Running verify-samples

Important: By default, samples must be pre-built before running verify-samples. Build the solution first, or pass --build to build samples during the run:

cd dotnet
dotnet build agent-framework-dotnet.slnx -f net10.0

Then run verify-samples:

# Run all samples across all categories
dotnet run --project eng/verify-samples -- --log results.log --csv results.csv

# Run a specific category
dotnet run --project eng/verify-samples -- --category 02-agents --log results.log

# Run specific samples by name
dotnet run --project eng/verify-samples -- Agent_Step02_StructuredOutput Agent_Step09_AsFunctionTool

# Control parallelism (default 8)
dotnet run --project eng/verify-samples -- --parallel 8 --log results.log

# Build samples during run (skips the need for a prior build step)
# This may cause build conflicts as multiple samples are built in parallel, so use with caution
dotnet run --project eng/verify-samples -- --build --log results.log

# Combine options
dotnet run --project eng/verify-samples -- --category 03-workflows --parallel 4 --log results.log --csv results.csv --md results.md

Required Environment Variables

The tool itself needs:

  • AZURE_OPENAI_ENDPOINT — for the AI verification agent
  • AZURE_OPENAI_DEPLOYMENT_NAME (optional, defaults to gpt-5-mini)

Individual samples require their own env vars (e.g., AZURE_AI_PROJECT_ENDPOINT). The tool automatically checks and skips samples with missing env vars.

Output Files

  • --log results.log — detailed per-sample log with stdout/stderr, AI reasoning, and a summary
  • --csv results.csv — tabular summary with Sample, ProjectPath, Status, FailedChecks, and Failures columns
  • --md results.md — Markdown summary with results table and collapsible failure details (suitable for GitHub PR comments)

Read the full file on GitHub · 228 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 228 lines · 40 tokens per session scan A dbec3e9727ff

Subscribe to this mod's changes

verify-samples-tool is a skill published in the GitHub repository microsoft/agent-framework (13,222 stars, last pushed 2d ago), licensed MIT. It adds 40 tokens to every session and 2,123 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

swarmclaw

AI agent runtime and multi-agent orchestration platform. Teaches agents how to use SwarmClaw's 6 primitive tools, persistent memory, dreaming, delegation, connectors, credentials, and the skill system. Use when an agent is running on SwarmClaw and needs to understand the platform's capabilities.

swarmclawai/swarmclaw · 67 tokens

agent-collaboration

Use this skill when coordinating multiple AI agents. Covers multi-agent patterns, handoffs, and orchestration strategies.

ApexIQ/skillsmith · 27 tokens

crewai-multi-agent

Multi-agent orchestration framework for autonomous AI collaboration. Use when building teams of specialized agents working together on complex tasks, when you need role-based agent collaboration with memory, or for production workflows requiring sequential/hierarchical execution. Built without LangChain dependencies…

davila7/claude-code-templates · 61 tokens

strands-review

Local preview of the strands-agents/devtools /strands review agent. Body is the upstream Task Reviewer SOP verbatim — do not paraphrase. Use when the user types /strands-review, asks for a "strands review" of a PR, or wants to anticipate what the remote /strands review GitHub Action will flag. Findings are close but…

strands-agents/harness-sdk · 125 tokens

docs-writer

Draft or rewrite Strands Agents documentation pages. Use when writing new doc pages, rewriting pages that failed audit, drafting sections for existing pages, or writing blog posts and release notes about Strands. Also triggers on "write a doc", "draft a page", "rewrite the quickstart", "add a tutorial for X"…

strands-agents/harness-sdk · 74 tokens

pr-writer

Generates pull request titles and descriptions. Use when the user asks to create, open, write, draft, or generate a PR, pull request, or merge request description.

strands-agents/harness-sdk · 39 tokens