Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/viktorbezdek/skillstack/agent-evaluationnpx skills add viktorbezdek/skillstack --skill agent-evaluationgit clone --depth 1 https://github.com/viktorbezdek/skillstackWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00115 | $0.03352 |
| Opus 5 | $0.00057 | $0.01676 |
| Sonnet 5 | $0.00023 | $0.00670 |
| Haiku 4.5 | $0.00012 | $0.00335 |
Grade A, and why
agent-evaluation scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 357 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Evaluating LLM Agent Systems
Agent evaluation requires fundamentally different approaches than traditional software testing. Agents make dynamic decisions, are non-deterministic, and often lack single correct answers. Effective evaluation must account for these characteristics while providing actionable feedback.
Key insight: LLM-as-a-Judge is not a single technique but a family of approaches, each suited to different evaluation contexts. Choosing the right approach and mitigating known biases is the core competency this skill develops.
When to Activate
- Testing agent performance systematically
- Validating context engineering choices
- Measuring improvements or catching regressions over time
- Building quality gates for agent pipelines
- Comparing different agent configurations or model outputs
- Building automated evaluation pipelines for LLM outputs
- Designing A/B tests for prompt or model changes
- Debugging evaluation systems that show inconsistent results
- Analyzing correlation between automated and human judgments
Decision Tree: Choosing an Evaluation Approach
What are you evaluating?
+-- Agent outputs against known correct answers?
| +-- Yes --> Direct Scoring (factual accuracy, format compliance, instruction following)
| +-- No --> Are you comparing two configurations?
| +-- Yes --> Pairwise Comparison with position-swap protocol
| | Criteria: tone, style, persuasiveness, creativity
| +-- No --> Do you have reference material?
| +-- Yes --> Reference-based evaluation (summarization, translation)
| +-- No --> Build rubrics first, then choose approach per dimension
Fundamentals
Why Agent Evaluation Is Different
Non-Determinism and Multiple Valid Paths: Agents may take different valid paths to reach goals. One agent searches three sources while another searches ten. Evaluate outcomes, not specific steps.
Context-Dependent Failures: Agent failures often depend on context in subtle ways. An agent might succeed on simple queries but fail on complex ones. Evaluation must cover a range of complexity levels.
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 357 lines · 115 tokens per session scan A c2cba90041df
agent-evaluation is a skill published in the GitHub repository viktorbezdek/skillstack (11 stars, last pushed 2mo ago), licensed MIT. It adds 115 tokens to every session and 3,352 once invoked, about $0.0006 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
gossipcat
Use when installing or setting up gossipcat multi-agent orchestration (parallel review, consensus, adaptive dispatch) in Claude Code or Cursor. Installs the gossipcat MCP server and hands off to gossipstatus() for all live rules.
Elephant Agent
Complete guide to what Elephant Agent is, how to use it from the CLI, how herd, wake, Personal Model, proactive curiosity, skills, and tools fit together, and how to explain Elephant Agent clearly to users, contributors, or evaluators.
libretranslate-markdown-i18n
Machine-translate Markdown / Obsidian / Quartz content into other languages using a self-hosted LibreTranslate instance, preserving frontmatter, headings, emojis, bold/italic, wikilinks (![[...]]/[[...]]) and internal links. Use whenever a user wants to auto-translate a Markdown vault, Obsidian notes, or a…
awesome-skills-master
Master catalog orchestrator and autonomous installer for AI agents. Use when exploring, cloning, discovering, or installing skills, plugins, or MCP servers from awesome-skills without using interactive CLI tools. Guides environment detection, rule adaptation (.mdc/CLAUDE.md), MCP configuration, and strict author…
readme-template
Standard README template for repos — professional structure with badges, overview, table of contents, features/modules, stack, installation, configuration, tests, security, structure, docs, roadmap, contribute, license, author + RepoActivity sections (Star History, repo stats) + profile GIF footer. Use when creating…
docker-single-port-multi-instance
Consolidate multiple Docker Compose app instances (production / test / demo) behind ONE host port using an nginx reverse proxy that routes by URL path prefix (e.g. /demo). Use when a user wants 'one port, several apps/banks' or a demo reachable simultaneously from the main landing page with its own database.