Borrowing it
Nothing to install: this file belongs to Galileo-Agent-Labs/eval-engineer. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/Galileo-Agent-Labs/eval-engineer/main/AGENTS.mdgit clone --depth 1 https://github.com/Galileo-Agent-Labs/eval-engineerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/instructions/galileo-agent-labs/eval-engineer/agents-md)<a href="https://agentmods.dev/instructions/galileo-agent-labs/eval-engineer/agents-md"><img src="https://agentmods.dev/badge/instructions/galileo-agent-labs/eval-engineer/agents-md/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/instructions/galileo-agent-labs/eval-engineer/agents-md"><img src="https://agentmods.dev/badge/instructions/galileo-agent-labs/eval-engineer/agents-md.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.01930 | $0.01930 |
| Opus 5.5 | $0.00772 | $0.00772 |
| Sonnet 5.5 | $0.00386 | $0.00386 |
| Haiku 4.5 | $0.00193 | $0.00193 |
Grade A, and why
eval-engineer AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 181 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Agent Instructions
This file is repo-local guidance for Codex, Claude Code, and other coding agents working on Eval Engineer. Keep it short and update it whenever the repo workflow, source-of-truth files, or recurring operating rules change.
Project Intent
Eval Engineer is a general Galileo evidence workflow for improving AI agents, RAG apps, and future AI systems. Do not shape the product around the first support-agent fixture. The support-agent cases are validation fixtures, not the skill's scope.
The near-term product goal is to reduce time-to-RCA for agent builders and owners while increasing Galileo discoverability for non-developer personas. Prefer RCA workflows that query log streams, inspect traces/sessions/spans, identify failure patterns, compare behavior over time, and return grounded answers with links or stable IDs back to Galileo data.
The north-star loop is:
- run the AI app
- log traces and metrics to Galileo
- fetch compact evidence
- diagnose failure
- propose a bounded fix
- verify with local and Galileo evidence
- keep only changes that improve measured behavior
Read First
docs/plan.md: product direction and architecture.docs/tasks.md: current task checklist and Linear issue mapping.docs/progress.md: latest work completed and next move..galileo/learnings.md: repo-specific durable learnings.blogs/: product thinking; useful for design intent, but do not treat as runtime instructions.
Skill Rules
- Canonical skill source:
skills/eval-engineer/. - Codex install link:
.agents/skills/eval-engineer. - Claude install link:
.claude/skills/eval-engineer. - Public installer CLI:
eval-engineerfrompyproject.toml. Keep it runnable throughuvx --from git+https://github.com/Galileo-Agent-Labs/eval-engineer.git. - Keep skill distribution skill-first. Future Codex and Claude plugins should package the canonical skill source instead of maintaining separate copies.
- Keep
SKILL.mdgeneral across agents, RAG, workflows, metrics, and providers. - Keep RCA outputs grounded in trace, span, session, metric, dataset, and experiment evidence.
- For reference fixtures, give each case a risk profile, quality dimensions, and case-specific Galileo metric profile. Do not rely on one global metric list to prove safety, quality, performance, and cost across all cases.
- Use
skills/eval-engineer/references/metric-profile-checklist.mdandskills/eval-engineer/assets/metric-profile-template.mdbefore optimizing cost or adding broad fixture coverage. - When logging test-suite cases to Galileo, include the full expected-output
contract in
dataset_output: expected decision, required/forbidden citations, required tools, answer constraints, abstention/permission requirements, risk profile, quality dimensions, and intended metrics. - Use
skills/eval-engineer/references/rca-recipe.mdfor generalized diagnose-fix-verify work and update it when a reusable Galileo RCA pattern is discovered. - When improving command skills, keep frontmatter descriptions trigger-focused,
keep
eval-engineeras a narrow router, load references conditionally, and promote recurring Galileo mistakes into focused skill gotchas and validation loops. - Keep detailed Galileo mechanics in
skills/eval-engineer/references/. - Keep deterministic helpers in
skills/eval-engineer/scripts/. - Do not hardcode
TC-1, the Nexus support agent, Brazil, one model, or one metric into the general skill.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 181 lines · 1,930 tokens per session scan A 86601652b5d8
eval-engineer AGENTS.md is an instructions file published in the GitHub repository Galileo-Agent-Labs/eval-engineer (43 stars, last pushed 2mo ago), licensed MIT. It adds 1,930 tokens to every session, about $0.0077 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other instructions, from other repositories
vscode buildNext.instructions.md
Working notes and architecture documentation for the new esbuild-based build system in build/next. Use when making changes to the new build pipeline (transpile/bundle commands, NLS plugin, source-map handling, resource copying, or self-hosting watch tasks).
next.js AGENTS.md
AGENTS.md instructions for vercel/next.js, covering next.js development guide, codebase structure, monorepo overview, core package: packages/next and other important packages.
vscode oss-third-party-notices.instructions.md
Instructions for microsoft/vscode, covering vs code oss third-party-notices pipeline, architecture, pipeline flow in ci, cache eligibility and rollout and applying the notice (cutover).
langchain AGENTS.md
AGENTS.md instructions for langchain-ai/langchain, covering global development guidelines for the langchain monorepo, corridor security analysis, project architecture and context, monorepo structure and development tools & commands.
deepseek-harness AGENTS.md
AGENTS.md instructions for deepseek-ai/deepseek-harness, covering agents.md, pre-stable apis and released session data, repository layout, commands and host sandbox failures.
vscode copilot-instructions.md
Copilot instructions for microsoft/vscode, covering vs code copilot instructions, project overview, root folders, core architecture (src/ folder) and built-in extensions (extensions/ folder).