implement-eval

A guide for adding scenario-based evaluations to the Prefect MCP server, which connects coding agents to Prefect.

In plain words
What is it for?
It is for creating test fixtures, writing evaluation prompts and checks, running focused and full evaluation suites, and updating the evaluation catalogue.
Why use it?
It helps test whether the agent gives the expected user-facing result for a real support or workflow question.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/prefecthq/prefect-mcp-server/implement-eval
Any agent
npx skills add PrefectHQ/prefect-mcp-server --skill implement-eval
Clone the repo
git clone --depth 1 https://github.com/PrefectHQ/prefect-mcp-server

Made for: Claude Code, Codex.

Per session 59 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 332 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00059 $0.00332
Opus 5 $0.00030 $0.00166
Sonnet 5 $0.00012 $0.00066
Haiku 4.5 $0.00006 $0.00033

Measured yesterday against content hash 4feb62410368, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

implement-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.agents/skills/implement-eval/SKILL.md · 24 lines

What it actually says

Implement a Prefect MCP eval

  1. Read AGENTS.md and evals/README.md before making changes.
  2. Resolve the requested scenario:
    • If the user provides a GitHub issue number or URL, read it with an available GitHub integration or the gh CLI.
    • Otherwise, use the scenario described by the user.
    • Identify the user-facing question, required Prefect state, expected investigation, and success criteria.
  3. Inspect related evals and fixtures before choosing a file location. Extend an existing scenario directory when appropriate; otherwise add a focused test under evals/.
  4. Implement the eval:
    • Create server state in a fixture.
    • Prompt the agent in language a Prefect user or support engineer would use.
    • Assert on the final behavior or answer, not incidental wording or private implementation details.
    • Keep protocol behavior in unit tests rather than evals.
  5. Run the narrowest relevant eval first, then run the full suite with just evals.
  6. Add or update the eval's row in evals/README.md.
  7. Review the diff for unrelated changes and report the scenario covered and verification performed.

Do not add live credentials, harness-specific argument placeholders, or client-specific tool syntax to the skill.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 24 lines · 59 tokens per session scan A 4feb62410368

Subscribe to this mod's changes

implement-eval is a skill published in the GitHub repository PrefectHQ/prefect-mcp-server (52 stars, last pushed 16d ago), licensed MIT. It adds 59 tokens to every session and 332 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

crewai-multi-agent

Multi-agent orchestration framework for autonomous AI collaboration. Use when building teams of specialized agents working together on complex tasks, when you need role-based agent collaboration with memory, or for production workflows requiring sequential/hierarchical execution. Built without LangChain dependencies…

davila7/claude-code-templates · 61 tokens

agent-framework-py-release

Use when cutting a Python release for the microsoft/agent-framework monorepo. Triggers on "bump py versions", "cut a python release", "prepare release PR for python", "release py packages", "bump python to X.Y.Z", or similar requests to bump Python package versions and prepare a release PR. Handles all four lifecycle…

microsoft/agent-framework · 103 tokens

python-package-management

Guide for managing packages in the Agent Framework Python monorepo, including creating new connector packages, versioning, and the lazy-loading pattern. Use this when adding, modifying, or releasing packages.

microsoft/agent-framework · 43 tokens

foundry-hosted-agent-validation

Step-by-step process for validating a Python Foundry hosted agent sample (under python/samples/04-hosting/foundry-hosted-agents/) end to end — running it locally (native runtime and azd ai agent run) and after deploying it to an Azure AI Foundry project with azd. Use this when asked to validate a hosted agent sample.

microsoft/agent-framework · 82 tokens

verify-samples-tool

How to use the verify-samples tool to run, verify, and manage sample definitions in the Agent Framework repository. Use this when adding, updating, or running sample verification.

microsoft/agent-framework · 40 tokens

build-and-test

How to build and test .NET projects in the Agent Framework repository. Use this when verifying or testing changes.

microsoft/agent-framework · 26 tokens