Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add Galileo-Agent-Labs/eval-engineer --skill eval-costgit clone --depth 1 https://github.com/Galileo-Agent-Labs/eval-engineerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-cost)<a href="https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-cost"><img src="https://agentmods.dev/badge/skills/galileo-agent-labs/eval-engineer/eval-cost/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-cost"><img src="https://agentmods.dev/badge/skills/galileo-agent-labs/eval-engineer/eval-cost.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00045 | $0.00628 |
| Opus 5.5 | $0.00018 | $0.00251 |
| Sonnet 5.5 | $0.00009 | $0.00126 |
| Haiku 4.5 | $0.00005 | $0.00063 |
Grade A, and why
eval-cost scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 67 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval Cost
Use this skill for tokenomics RCA. Cost changes are accepted only when Galileo quality metrics do not regress.
Conditional References
- Load
skills/eval-engineer/references/tokenomics-rca.mdwhen choosing the tokenomics workflow or diagnosing why cost moved. - Run
skills/eval-engineer/scripts/compare_tokenomics_packets.pywhen both baseline and verification packets exist. - Load
skills/eval-engineer/assets/cost-diagnosis-template.md,skills/eval-engineer/assets/tokenomics-fix-plan-template.md, andskills/eval-engineer/assets/quality-preserving-verification-template.mdonly when writing those artifacts.
Do
- Compare cost, latency, tokens, retrieved context, tool calls, retries, rerank/self-check spans, model spans, and evaluator cost.
- Run
compare_tokenomics_packets.pywithout explicit quality metrics first when packets use custom quality names, then inspect the inferredQuality metrics comparedlist before accepting the decision. - Treat behavior counters such as handoff count, tool count, step count, retry count, and self-check count as efficiency or workflow evidence, not quality gates by default. Promote one to quality only when the metric profile states the desired direction for that route or segment.
- Protect named quality metrics and segment gates.
- Reject cheaper candidates when aggregate quality holds but any required segment gate regresses.
- Treat lower traffic volume as inconclusive unless per-trace efficiency also improves.
Gotchas
- Cost, latency, wall time, duration, token count, span count, and tool count are efficiency evidence, not quality gates by default.
- Quality metrics are not always higher-is-better. Error, toxicity, hallucination, policy-violation, and tool-error rates regress when they rise.
- Aggregate quality can hide route, risk-profile, or customer-segment regressions.
- For RAG pruning, include hard multi-source or multi-hop cases before keeping a top-k reduction.
- For agentic workflows, compare agent steps, planner spans, rerank passes, self-check spans, and tool calls so the cheaper loop is explainable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 67 lines · 45 tokens per session scan A 01ece65f2c65
eval-cost is a skill published in the GitHub repository Galileo-Agent-Labs/eval-engineer (43 stars, last pushed 2mo ago), licensed MIT. It adds 45 tokens to every session and 628 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
rag-evaluation
Comprehensive RAG evaluation with retrieval metrics, generation quality, and end-to-end testing. Use this skill when measuring and improving RAG system performance. Activate when: RAG evaluation, RAGAS, retrieval metrics, generation quality, RAG testing, MRR, recall, faithfulness.
llm-fallback-chains
Use this skill when implementing fallback strategies for LLM applications. Activate when the user needs graceful degradation for AI services, multi-provider failover, handling LLM outages, or building resilient AI systems.
agentic-rag
Build autonomous RAG agents that reason, plan, and use tools for complex retrieval tasks. Use this skill when simple retrieve-and-generate isn't enough. Activate when: agentic RAG, RAG agent, multi-step retrieval, tool-using RAG, autonomous retrieval, query decomposition.
corrective-rag
Implement Corrective RAG (CRAG) with retrieval validation, fallback strategies, and self-correction. Use this skill when RAG outputs need quality guarantees and automatic error correction. Activate when: CRAG, corrective RAG, retrieval validation, fallback search, self-correcting RAG, grounded generation.
graphrag-patterns
Implement GraphRAG patterns combining knowledge graphs with retrieval for complex reasoning. Use this skill when building RAG over interconnected data or needing relationship-aware retrieval. Activate when: GraphRAG, knowledge graph, graph retrieval, entity relationships, Neo4j RAG, graph database, connected data.
production-rag-checklist
Comprehensive checklist for deploying RAG systems to production with reliability and scale. Use this skill when preparing RAG for production deployment. Activate when: production RAG, RAG deployment, RAG checklist, RAG scaling, RAG monitoring, production-ready RAG.