Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add beel-collab/presets.dev --skill evaluationgit clone --depth 1 https://github.com/beel-collab/presets.devWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/beel-collab/presets.dev/evaluation)<a href="https://agentmods.dev/skills/beel-collab/presets.dev/evaluation"><img src="https://agentmods.dev/badge/skills/beel-collab/presets.dev/evaluation/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/beel-collab/presets.dev/evaluation"><img src="https://agentmods.dev/badge/skills/beel-collab/presets.dev/evaluation.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00025 | $0.02048 |
| Opus 5.5 | $0.00010 | $0.00819 |
| Sonnet 5.5 | $0.00005 | $0.00410 |
| Haiku 4.5 | $0.00003 | $0.00205 |
Grade A, and why
evaluation scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
Copies of this mod
1 near-identical copy found in the catalogue:
- evaluation — 94% identical, 13 lines differ
How it starts
The opening of the file, as written. The whole thing — 245 lines — stays where its author put it; the contents beside it link to each section on GitHub.
When to Use This Skill
Build evaluation frameworks for agent systems
Use this skill when working with build evaluation frameworks for agent systems.
Evaluation Methods for Agent Systems
Evaluation of agent systems requires different approaches than traditional software or even standard language model applications. Agents make dynamic decisions, are non-deterministic between runs, and often lack single correct answers. Effective evaluation must account for these characteristics while providing actionable feedback. A robust evaluation framework enables continuous improvement, catches regressions, and validates that context engineering choices achieve intended effects.
When to Use
Activate this skill when:
- Testing agent performance systematically
- Validating context engineering choices
- Measuring improvements over time
- Catching regressions before deployment
- Building quality gates for agent pipelines
- Comparing different agent configurations
- Evaluating production systems continuously
Core Concepts
Agent evaluation requires outcome-focused approaches that account for non-determinism and multiple valid paths. Multi-dimensional rubrics capture various quality aspects: factual accuracy, completeness, citation accuracy, source quality, and tool efficiency. LLM-as-judge provides scalable evaluation while human evaluation catches edge cases.
The key insight is that agents may find alternative paths to goals—the evaluation should judge whether they achieve right outcomes while following reasonable processes.
Performance Drivers: The 95% Finding Research on the BrowseComp evaluation (which tests browsing agents' ability to locate hard-to-find information) found that three factors explain 95% of performance variance:
| Factor | Variance Explained | Implication |
|---|---|---|
| Token usage | 80% | More tokens = better performance |
| Number of tool calls | ~10% | More exploration helps |
| Model choice | ~5% | Better models multiply efficiency |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 245 lines · 25 tokens per session scan A 4f6870d06e6b
evaluation is a skill published in the GitHub repository beel-collab/presets.dev (3 stars, last pushed 5mo ago), licensed MIT. It adds 25 tokens to every session and 2,048 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
deepeval
Use when discussing or working with DeepEval (the python AI evaluation framework).
iterative-refinement
Disciplined, measurable iteration for a substantial refinement or investigation: loop against verifiable pass/fail conditions, fan work out to subagents, and keep the main context lean. Use when improving something measurable over repeated cycles (tuning a metric or detector, refactoring against a regression bar)…
diagnose-hard-problem
Disciplined diagnosis loop for hard problems, diagnosing bugs and regressions. Reproduce → minimise → hypothesise → instrument → fix → regression-test.
skill-creator-primer
You MUST load this skill before the skill-creator skill AND before making ANY change to, or conducting a review of ANY Agent Skill. Triggers include creating, editing, reviewing, or contributing to any part of an Agent Skill (description, frontmatter, body, references, scripts, trigger evals, conflicts, etc).
llm-wiki
Use when building or maintaining a self-contained personal knowledge base (an LLM wiki) in plain markdown. Triggers: ingesting sources into a wiki, querying wiki knowledge, linting wiki health, auditing article claims against their sources, critiquing a wiki source's reasoning, superseding stale knowledge, 'add to…
apply-mantel-styles
Provides guidelines for applying Mantel's brand styles to diagrams and frontend components. Use when asked to create visuals that need to align with Mantel's branding.