Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add Galileo-Agent-Labs/eval-engineer --skill eval-diagnosegit clone --depth 1 https://github.com/Galileo-Agent-Labs/eval-engineerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-diagnose)<a href="https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-diagnose"><img src="https://agentmods.dev/badge/skills/galileo-agent-labs/eval-engineer/eval-diagnose.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00041 | $0.00584 |
| Opus 5 | $0.00020 | $0.00292 |
| Sonnet 5 | $0.00008 | $0.00117 |
| Haiku 4.5 | $0.00004 | $0.00058 |
Grade A, and why
eval-diagnose scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 62 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval Diagnose
Use this skill for evidence-backed RCA once a packet, URL-derived evidence, or trace/session/log-stream context is available.
Required Reference
Use skills/eval-engineer/references/rca-recipe.md,
skills/eval-engineer/references/debug-packets.md,
skills/eval-engineer/references/evidence-provenance.md, and
skills/eval-engineer/assets/diagnosis-template.md.
Do
- Start from fetched evidence, not source-code guesses.
- Name the failing metric contract and what it proves.
- Label hosted Galileo evidence separately from local deterministic packets before making metric or score claims.
- Inspect traces, spans, sessions, tool calls, retrieval context, and scorer status to classify the fix surface.
- Classify the fix surface: prompt, tool schema, adapter, retriever, ranker, guardrail, metric, dataset, or SDK wiring.
- Write diagnosis and bounded fix plan only when evidence supports it.
- Honor read-only requests. If the user says read-only, dry run, no edits, or
"do not edit files", do not write
.galileo/artifacts. Return the RCA inline and include a short "Would write" list for any suggested artifact paths.
Gotchas
- Fetched debug packets are the RCA source of truth when scorer jobs are still settling or runner output disagrees with fetched metrics.
- A prompt diff, local score, or code diff is not proof of improvement without before/after Galileo evidence.
- Bare correctness or factuality can be a smoke test only. Prefer the metric contract tied to the case risk.
- Safe final wording is not enough for source-authority cases if unsafe or forbidden retrieved context entered the task surface.
Validation Loop
Before finalizing:
- Run
python3 skills/eval-engineer/scripts/summarize_debug_packet.py <packet>unless a compact summary is already available. - Check that every RCA claim names supporting metric, trace, span, session, experiment, dataset, or log-stream evidence.
- Confirm the fix surface is bounded and allowed by
.galileo/config.yml. - If any claim lacks evidence, downgrade it to a hypothesis or route to
/eval-fetchor/eval-measure.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago First seen · 62 lines · 41 tokens per session scan A 40c50e7e3f67
eval-diagnose is a skill published in the GitHub repository Galileo-Agent-Labs/eval-engineer (41 stars, last pushed 26d ago), licensed MIT. It adds 41 tokens to every session and 584 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
agent-checkpointing
Implement checkpointing for agent recovery, debugging, and replay. Use this skill when building recoverable agents, implementing replay, debugging agent failures, or creating resumable workflows. Activate when: agent checkpoint, agent recovery, resume agent, agent restart, workflow replay, agent debugging, failure…
error-pattern-analyzer
Use this skill when analyzing error patterns in applications. Activate when the user has recurring errors, wants to find root causes of issues, needs to identify systemic problems, is analyzing error logs, or wants to categorize and prioritize bugs.
log-forensics
Use this skill when investigating issues through logs. Activate when the user needs to analyze log files, search for specific events in logs, correlate logs across services, investigate incidents through logs, or extract insights from application logs.
performance-profiler
Use this skill when investigating performance issues. Activate when the user has slow code, needs to find performance bottlenecks, wants to profile application performance, is optimizing response times, or investigating memory usage.
stack-trace-decoder
Use this skill when analyzing stack traces. Activate when the user has a stack trace to understand, needs to decode error traces, wants to find the root cause from a stack trace, is debugging crashes, or needs help interpreting exception traces.
metrics-logs-traces
Implement comprehensive observability with metrics, logs, and distributed traces. Use this skill when setting up monitoring, debugging production issues, or implementing observability. Activate when: metrics, logs, traces, observability, monitoring, Datadog, Prometheus, Grafana, OpenTelemetry, distributed tracing…