Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add Galileo-Agent-Labs/eval-engineer --skill eval-measuregit clone --depth 1 https://github.com/Galileo-Agent-Labs/eval-engineerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-measure)<a href="https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-measure"><img src="https://agentmods.dev/badge/skills/galileo-agent-labs/eval-engineer/eval-measure.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00036 | $0.00423 |
| Opus 5 | $0.00018 | $0.00211 |
| Sonnet 5 | $0.00007 | $0.00085 |
| Haiku 4.5 | $0.00004 | $0.00042 |
Grade A, and why
eval-measure scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
What it actually says
Eval Measure
Use this skill before optimization or broad fixture work. Its job is to make the measurement contract explicit.
Required Reference
Use skills/eval-engineer/references/metric-profile-checklist.md and
skills/eval-engineer/assets/metric-profile-template.md.
Do
- Define risk profile and quality dimensions.
- Write the full expected-output contract: expected decision, required and forbidden citations, tools, answer constraints, abstention, permissions, and safety requirements.
- Include retrieved-source gates when source authority matters:
required_retrieved_sources,forbidden_retrieved_sources, and whether final citations alone are sufficient for the case risk. - Prefer independent observations over agent self-reports for safety gates. Check answer text, citations, retrieved source IDs, tool calls, and Galileo scorers before accepting flags emitted by the app under test.
- Choose Galileo metrics by failure contract, not by one global list.
- Identify metric gaps before accepting a cost or quality change.
Gotchas
- Agent self-reported flags are instrumentation breadcrumbs, not acceptance evidence.
- Final citations alone are not enough when retrieved-source authority is part of the risk.
- Do not infer latency, wall time, tokens, or cost as quality metrics.
Validation Loop
Before finalizing a metric profile, check it against
skills/eval-engineer/references/metric-profile-checklist.md and confirm the
profile names quality gates, cost/performance metrics, segment gates, metric
direction, and known gaps.
Output
Findings first. Start with the highest-risk metric gaps and the keep/reject/ inconclusive measurement decision, then produce a metric profile or explain what evidence is missing. Do not improve the app until the expected-output contract and acceptance gates are clear.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago First seen · 52 lines · 36 tokens per session scan A 058efcb6ed5f
eval-measure is a skill published in the GitHub repository Galileo-Agent-Labs/eval-engineer (41 stars, last pushed 27d ago), licensed MIT. It adds 36 tokens to every session and 423 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
rag-evaluation
Comprehensive RAG evaluation with retrieval metrics, generation quality, and end-to-end testing. Use this skill when measuring and improving RAG system performance. Activate when: RAG evaluation, RAGAS, retrieval metrics, generation quality, RAG testing, MRR, recall, faithfulness.
llm-fallback-chains
Use this skill when implementing fallback strategies for LLM applications. Activate when the user needs graceful degradation for AI services, multi-provider failover, handling LLM outages, or building resilient AI systems.
agentic-rag
Build autonomous RAG agents that reason, plan, and use tools for complex retrieval tasks. Use this skill when simple retrieve-and-generate isn't enough. Activate when: agentic RAG, RAG agent, multi-step retrieval, tool-using RAG, autonomous retrieval, query decomposition.
corrective-rag
Implement Corrective RAG (CRAG) with retrieval validation, fallback strategies, and self-correction. Use this skill when RAG outputs need quality guarantees and automatic error correction. Activate when: CRAG, corrective RAG, retrieval validation, fallback search, self-correcting RAG, grounded generation.
graphrag-patterns
Implement GraphRAG patterns combining knowledge graphs with retrieval for complex reasoning. Use this skill when building RAG over interconnected data or needing relationship-aware retrieval. Activate when: GraphRAG, knowledge graph, graph retrieval, entity relationships, Neo4j RAG, graph database, connected data.
production-rag-checklist
Comprehensive checklist for deploying RAG systems to production with reliability and scale. Use this skill when preparing RAG for production deployment. Activate when: production RAG, RAG deployment, RAG checklist, RAG scaling, RAG monitoring, production-ready RAG.