Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add instructions/galileo-agent-labs/eval-engineer/agents-mdgit clone --depth 1 https://github.com/Galileo-Agent-Labs/eval-engineerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/instructions/galileo-agent-labs/eval-engineer/agents-md)<a href="https://agentmods.dev/instructions/galileo-agent-labs/eval-engineer/agents-md"><img src="https://agentmods.dev/badge/instructions/galileo-agent-labs/eval-engineer/agents-md.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.01930 | $0.01930 |
| Opus 5 | $0.00965 | $0.00965 |
| Sonnet 5 | $0.00386 | $0.00386 |
| Haiku 4.5 | $0.00193 | $0.00193 |
Grade A, and why
eval-engineer AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 181 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Agent Instructions
This file is repo-local guidance for Codex, Claude Code, and other coding agents working on Eval Engineer. Keep it short and update it whenever the repo workflow, source-of-truth files, or recurring operating rules change.
Project Intent
Eval Engineer is a general Galileo evidence workflow for improving AI agents, RAG apps, and future AI systems. Do not shape the product around the first support-agent fixture. The support-agent cases are validation fixtures, not the skill's scope.
The near-term product goal is to reduce time-to-RCA for agent builders and owners while increasing Galileo discoverability for non-developer personas. Prefer RCA workflows that query log streams, inspect traces/sessions/spans, identify failure patterns, compare behavior over time, and return grounded answers with links or stable IDs back to Galileo data.
The north-star loop is:
- run the AI app
- log traces and metrics to Galileo
- fetch compact evidence
- diagnose failure
- propose a bounded fix
- verify with local and Galileo evidence
- keep only changes that improve measured behavior
Read First
docs/plan.md: product direction and architecture.docs/tasks.md: current task checklist and Linear issue mapping.docs/progress.md: latest work completed and next move..galileo/learnings.md: repo-specific durable learnings.blogs/: product thinking; useful for design intent, but do not treat as runtime instructions.
Skill Rules
- Canonical skill source:
skills/eval-engineer/. - Codex install link:
.agents/skills/eval-engineer. - Claude install link:
.claude/skills/eval-engineer. - Public installer CLI:
eval-engineerfrompyproject.toml. Keep it runnable throughuvx --from git+https://github.com/Galileo-Agent-Labs/eval-engineer.git. - Keep skill distribution skill-first. Future Codex and Claude plugins should package the canonical skill source instead of maintaining separate copies.
- Keep
SKILL.mdgeneral across agents, RAG, workflows, metrics, and providers. - Keep RCA outputs grounded in trace, span, session, metric, dataset, and experiment evidence.
- For reference fixtures, give each case a risk profile, quality dimensions, and case-specific Galileo metric profile. Do not rely on one global metric list to prove safety, quality, performance, and cost across all cases.
- Use
skills/eval-engineer/references/metric-profile-checklist.mdandskills/eval-engineer/assets/metric-profile-template.mdbefore optimizing cost or adding broad fixture coverage. - When logging test-suite cases to Galileo, include the full expected-output
contract in
dataset_output: expected decision, required/forbidden citations, required tools, answer constraints, abstention/permission requirements, risk profile, quality dimensions, and intended metrics. - Use
skills/eval-engineer/references/rca-recipe.mdfor generalized diagnose-fix-verify work and update it when a reusable Galileo RCA pattern is discovered. - When improving command skills, keep frontmatter descriptions trigger-focused,
keep
eval-engineeras a narrow router, load references conditionally, and promote recurring Galileo mistakes into focused skill gotchas and validation loops. - Keep detailed Galileo mechanics in
skills/eval-engineer/references/. - Keep deterministic helpers in
skills/eval-engineer/scripts/. - Do not hardcode
TC-1, the Nexus support agent, Brazil, one model, or one metric into the general skill.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 181 lines · 1,930 tokens per session scan A 86601652b5d8
eval-engineer AGENTS.md is an instructions file published in the GitHub repository Galileo-Agent-Labs/eval-engineer (41 stars, last pushed 23d ago), licensed MIT. It adds 1,930 tokens to every session, about $0.0097 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other instructions, from other repositories
agent-skills AGENTS.md
AGENTS.md instructions for magnus919/agent-skills, covering agents.md — agent guide for agent-skills, format compliance, readme format, required sections and style guidance.
gangsta AGENTS.md
Instructions for kucherenko/gangsta, covering contributing to gangsta, what this is, what will not be accepted, skill changes and the laws apply to contributors too.
shisan-xinuo-workflow AGENTS.md
AGENTS.md instructions for zxc663/shisan-xinuo-workflow, covering 项目级 agent 规则 · shisan-xinuo-workflow（本仓库）, 工作流在场（本仓库会话）, 维护纪律（本 skill 正文 vs 史料——开发本 skill 时的写作规范）, 项目承载（已就绪） and 本仓库底线（区别于通用纪律）.
gangsta GEMINI.md
Instructions for kucherenko/gangsta, a project described as: AI agentic skills framework for spec-driven development, built on the organizational model of mafia.
agent-skills AGENTS.md
AGENTS.md instructions for racastellanosm/agent-skills, covering agents.md, 1. vision & architectural invariants, 2. repository architecture & layout, 3. harness support & installation paths and 4. scripting & tooling guardrails.
connectors CLAUDE.md
Claude Code instructions for zapier/connectors: Claude Code loads this file automatically. The canonical AI-agent guidance lives in AGENTS.md so that Claude Code, Codex, Cursor, and any other tool following the AGENTS.md convention read the same content.