Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/galileo-agent-labs/eval-engineer/eval-costnpx skills add Galileo-Agent-Labs/eval-engineer --skill eval-costgit clone --depth 1 https://github.com/Galileo-Agent-Labs/eval-engineerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-cost)<a href="https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-cost"><img src="https://agentmods.dev/badge/skills/galileo-agent-labs/eval-engineer/eval-cost.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00045 | $0.00628 |
| Opus 5 | $0.00023 | $0.00314 |
| Sonnet 5 | $0.00009 | $0.00126 |
| Haiku 4.5 | $0.00005 | $0.00063 |
Grade A, and why
eval-cost scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 67 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval Cost
Use this skill for tokenomics RCA. Cost changes are accepted only when Galileo quality metrics do not regress.
Conditional References
- Load
skills/eval-engineer/references/tokenomics-rca.mdwhen choosing the tokenomics workflow or diagnosing why cost moved. - Run
skills/eval-engineer/scripts/compare_tokenomics_packets.pywhen both baseline and verification packets exist. - Load
skills/eval-engineer/assets/cost-diagnosis-template.md,skills/eval-engineer/assets/tokenomics-fix-plan-template.md, andskills/eval-engineer/assets/quality-preserving-verification-template.mdonly when writing those artifacts.
Do
- Compare cost, latency, tokens, retrieved context, tool calls, retries, rerank/self-check spans, model spans, and evaluator cost.
- Run
compare_tokenomics_packets.pywithout explicit quality metrics first when packets use custom quality names, then inspect the inferredQuality metrics comparedlist before accepting the decision. - Treat behavior counters such as handoff count, tool count, step count, retry count, and self-check count as efficiency or workflow evidence, not quality gates by default. Promote one to quality only when the metric profile states the desired direction for that route or segment.
- Protect named quality metrics and segment gates.
- Reject cheaper candidates when aggregate quality holds but any required segment gate regresses.
- Treat lower traffic volume as inconclusive unless per-trace efficiency also improves.
Gotchas
- Cost, latency, wall time, duration, token count, span count, and tool count are efficiency evidence, not quality gates by default.
- Quality metrics are not always higher-is-better. Error, toxicity, hallucination, policy-violation, and tool-error rates regress when they rise.
- Aggregate quality can hide route, risk-profile, or customer-segment regressions.
- For RAG pruning, include hard multi-source or multi-hop cases before keeping a top-k reduction.
- For agentic workflows, compare agent steps, planner spans, rerank passes, self-check spans, and tool calls so the cheaper loop is explainable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 6d ago First seen · 67 lines · 45 tokens per session scan A 01ece65f2c65
eval-cost is a skill published in the GitHub repository Galileo-Agent-Labs/eval-engineer (41 stars, last pushed 25d ago), licensed MIT. It adds 45 tokens to every session and 628 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
rag-evaluation
Comprehensive RAG evaluation with retrieval metrics, generation quality, and end-to-end testing. Use this skill when measuring and improving RAG system performance. Activate when: RAG evaluation, RAGAS, retrieval metrics, generation quality, RAG testing, MRR, recall, faithfulness.
llm-fallback-chains
Use this skill when implementing fallback strategies for LLM applications. Activate when the user needs graceful degradation for AI services, multi-provider failover, handling LLM outages, or building resilient AI systems.
corrective-rag
Implement Corrective RAG (CRAG) with retrieval validation, fallback strategies, and self-correction. Use this skill when RAG outputs need quality guarantees and automatic error correction. Activate when: CRAG, corrective RAG, retrieval validation, fallback search, self-correcting RAG, grounded generation.
graphrag-patterns
Implement GraphRAG patterns combining knowledge graphs with retrieval for complex reasoning. Use this skill when building RAG over interconnected data or needing relationship-aware retrieval. Activate when: GraphRAG, knowledge graph, graph retrieval, entity relationships, Neo4j RAG, graph database, connected data.
llm-rate-limiting
Use this skill when implementing rate limiting for LLM APIs. Activate when the user needs to prevent API quota exhaustion, implement backoff strategies, handle rate limit errors, or manage concurrent LLM requests.
model-routing-strategy
Use this skill when implementing model selection for LLM applications. Activate when the user needs to choose between different AI models, implement cost-efficient model routing, balance quality vs cost, or build intelligent model selection systems.