Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add viktorbezdek/skillstack --skill agent-evaluationgit clone --depth 1 https://github.com/viktorbezdek/skillstackWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/viktorbezdek/skillstack/agent-evaluation)<a href="https://agentmods.dev/skills/viktorbezdek/skillstack/agent-evaluation"><img src="https://agentmods.dev/badge/skills/viktorbezdek/skillstack/agent-evaluation/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/viktorbezdek/skillstack/agent-evaluation"><img src="https://agentmods.dev/badge/skills/viktorbezdek/skillstack/agent-evaluation.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00115 | $0.03352 |
| Opus 5.5 | $0.00046 | $0.01341 |
| Sonnet 5.5 | $0.00023 | $0.00670 |
| Haiku 4.5 | $0.00012 | $0.00335 |
Grade A, and why
agent-evaluation scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 357 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Evaluating LLM Agent Systems
Agent evaluation requires fundamentally different approaches than traditional software testing. Agents make dynamic decisions, are non-deterministic, and often lack single correct answers. Effective evaluation must account for these characteristics while providing actionable feedback.
Key insight: LLM-as-a-Judge is not a single technique but a family of approaches, each suited to different evaluation contexts. Choosing the right approach and mitigating known biases is the core competency this skill develops.
When to Activate
- Testing agent performance systematically
- Validating context engineering choices
- Measuring improvements or catching regressions over time
- Building quality gates for agent pipelines
- Comparing different agent configurations or model outputs
- Building automated evaluation pipelines for LLM outputs
- Designing A/B tests for prompt or model changes
- Debugging evaluation systems that show inconsistent results
- Analyzing correlation between automated and human judgments
Decision Tree: Choosing an Evaluation Approach
What are you evaluating?
+-- Agent outputs against known correct answers?
| +-- Yes --> Direct Scoring (factual accuracy, format compliance, instruction following)
| +-- No --> Are you comparing two configurations?
| +-- Yes --> Pairwise Comparison with position-swap protocol
| | Criteria: tone, style, persuasiveness, creativity
| +-- No --> Do you have reference material?
| +-- Yes --> Reference-based evaluation (summarization, translation)
| +-- No --> Build rubrics first, then choose approach per dimension
Fundamentals
Why Agent Evaluation Is Different
Non-Determinism and Multiple Valid Paths: Agents may take different valid paths to reach goals. One agent searches three sources while another searches ten. Evaluate outcomes, not specific steps.
Context-Dependent Failures: Agent failures often depend on context in subtle ways. An agent might succeed on simple queries but fail on complex ones. Evaluation must cover a range of complexity levels.
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 357 lines · 115 tokens per session scan A c2cba90041df
agent-evaluation is a skill published in the GitHub repository viktorbezdek/skillstack (14 stars, last pushed 4mo ago), licensed MIT. It adds 115 tokens to every session and 3,352 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
agent-qa-authoring
Create, edit, validate, and run Agent QA tests, suites, and hooks through MCP or CLI while preserving canonical IDs and schema contracts.
agent-harness-fault-injection
Use when an agent workflow needs deterministic recovery evidence for sandbox, MCP/tool, worker, checkpoint, memory, or orchestration failures.
agent-evaluation
Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt changes, reproducing failures, or running agent regression tests.
agent-evaluation-reporting
Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.
android-ui-journey-testing
XML-specified Android UI journey testing, interactive step execution, assertion verification, and JSON outcome reporting.
quinn
Proves the system works by writing and executing comprehensive test suites.