Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add Galileo-Agent-Labs/eval-engineer --skill eval-datasetgit clone --depth 1 https://github.com/Galileo-Agent-Labs/eval-engineerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-dataset)<a href="https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-dataset"><img src="https://agentmods.dev/badge/skills/galileo-agent-labs/eval-engineer/eval-dataset/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-dataset"><img src="https://agentmods.dev/badge/skills/galileo-agent-labs/eval-engineer/eval-dataset.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00041 | $0.00614 |
| Opus 5.5 | $0.00016 | $0.00246 |
| Sonnet 5.5 | $0.00008 | $0.00123 |
| Haiku 4.5 | $0.00004 | $0.00061 |
Grade A, and why
eval-dataset scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 67 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval Dataset
Use this skill to build durable eval cases from evidence. Its job is dataset quality control, not diagnosis or app fixing.
Required Reference
Use skills/eval-engineer/references/eval-datasets.md for the canonical case
schema precedence, optional review metadata, promotion rules, bootstrap
guidance for different use cases, and Galileo SDK dataset usage.
Required Inputs
Start from at least one evidence source:
.galileo/current/debug-packet.json.galileo/current/diagnosis.md- Galileo trace, session, experiment, or log-stream IDs
- production symptom with enough context to define expected behavior
If there is no concrete failure, regression, policy requirement, or metric gap, ask for evidence before writing a case.
Do
- Write new unreviewed cases to
.galileo/eval-dataset/candidates.jsonlunless the user asks to accept or reject a case. - When accepting or rejecting cases, update
.galileo/eval-dataset/changelog.md. - Bootstrap datasets by use case: RAG, tool-calling agent, multi-turn, workflow, safety/compliance, and tokenomics.
- Choose failure triggers that should be caught by named Galileo metrics or explicit local gates.
- Follow the user-provided schema when the user gives one.
- Follow the existing Galileo dataset schema when appending to a fixed dataset.
- Do not force Eval Engineer fields into an upload schema. Put optional review
metadata in
.galileo/eval-dataset/candidates.jsonl,metadata, or a sidecar file when needed. - Use retrieved-source gates when source authority matters and the chosen schema can carry them. Final citations alone are not enough for prompt-injection, privacy, stale-source, or source-authority cases.
- When the user wants to upload, reuse, or run cases in Galileo, use the SDK workflow in the reference instead of improvising dataset shapes.
- Do not promote candidates without human review.
Validation Loop
Before finalizing a dataset change:
- Check the case against
skills/eval-engineer/references/eval-datasets.md. - Confirm the case has a concrete failure trigger and metric or local gate that should catch it.
- Confirm required/forbidden citations, tools, permissions, source gates, and abstention rules are explicit when relevant.
- Validate JSONL syntax after writing or editing.
- If accepting or rejecting a case, update
.galileo/eval-dataset/changelog.md.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 67 lines · 41 tokens per session scan A 9ced085623e0
eval-dataset is a skill published in the GitHub repository Galileo-Agent-Labs/eval-engineer (43 stars, last pushed 2mo ago), licensed MIT. It adds 41 tokens to every session and 614 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
agent-testing-harness
Use this skill when testing AI agent systems. Activate when the user needs to test agent behavior, write tests for multi-agent systems, implement agent evaluation frameworks, create test harnesses for autonomous agents, or validate agent outputs systematically.
test-generation-patterns
Use this skill when generating tests with AI assistance. Activate when the user wants to create unit tests, integration tests, generate test cases, improve test coverage, write tests for existing code, or set up testing patterns for their project.
api-test-patterns
Write comprehensive API tests for REST and GraphQL endpoints. Use this skill when testing APIs, writing contract tests, or validating integrations. Activate when: api testing, REST test, GraphQL test, endpoint testing, integration test, postman, contract testing.
playwright-patterns
Write reliable, maintainable E2E tests with Playwright best practices. Use this skill when writing Playwright tests, debugging flaky tests, or setting up E2E automation. Activate when: playwright, e2e test, end-to-end, browser testing, UI automation, web testing.
test-case-design
Design comprehensive test cases with proper coverage and clear documentation. Use this skill when writing test cases, creating test suites, or improving test coverage. Activate when: test case, test scenario, test coverage, test design, write tests, BDD, gherkin.
skill-testing
Test skills for correct activation, content quality, and regression — both automated checks (frontmatter validity, lint) and manual verification (query-suite activation testing). Covers CI integration and how to catch skill regressions before users do. Use this skill when adding skills to a repo, setting up CI for a…