Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/galileo-agent-labs/eval-engineer/eval-datasetnpx skills add Galileo-Agent-Labs/eval-engineer --skill eval-datasetgit clone --depth 1 https://github.com/Galileo-Agent-Labs/eval-engineerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-dataset)<a href="https://agentmods.dev/skills/galileo-agent-labs/eval-engineer/eval-dataset"><img src="https://agentmods.dev/badge/skills/galileo-agent-labs/eval-engineer/eval-dataset.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00041 | $0.00614 |
| Opus 5 | $0.00020 | $0.00307 |
| Sonnet 5 | $0.00008 | $0.00123 |
| Haiku 4.5 | $0.00004 | $0.00061 |
Grade A, and why
eval-dataset scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 67 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval Dataset
Use this skill to build durable eval cases from evidence. Its job is dataset quality control, not diagnosis or app fixing.
Required Reference
Use skills/eval-engineer/references/eval-datasets.md for the canonical case
schema precedence, optional review metadata, promotion rules, bootstrap
guidance for different use cases, and Galileo SDK dataset usage.
Required Inputs
Start from at least one evidence source:
.galileo/current/debug-packet.json.galileo/current/diagnosis.md- Galileo trace, session, experiment, or log-stream IDs
- production symptom with enough context to define expected behavior
If there is no concrete failure, regression, policy requirement, or metric gap, ask for evidence before writing a case.
Do
- Write new unreviewed cases to
.galileo/eval-dataset/candidates.jsonlunless the user asks to accept or reject a case. - When accepting or rejecting cases, update
.galileo/eval-dataset/changelog.md. - Bootstrap datasets by use case: RAG, tool-calling agent, multi-turn, workflow, safety/compliance, and tokenomics.
- Choose failure triggers that should be caught by named Galileo metrics or explicit local gates.
- Follow the user-provided schema when the user gives one.
- Follow the existing Galileo dataset schema when appending to a fixed dataset.
- Do not force Eval Engineer fields into an upload schema. Put optional review
metadata in
.galileo/eval-dataset/candidates.jsonl,metadata, or a sidecar file when needed. - Use retrieved-source gates when source authority matters and the chosen schema can carry them. Final citations alone are not enough for prompt-injection, privacy, stale-source, or source-authority cases.
- When the user wants to upload, reuse, or run cases in Galileo, use the SDK workflow in the reference instead of improvising dataset shapes.
- Do not promote candidates without human review.
Validation Loop
Before finalizing a dataset change:
- Check the case against
skills/eval-engineer/references/eval-datasets.md. - Confirm the case has a concrete failure trigger and metric or local gate that should catch it.
- Confirm required/forbidden citations, tools, permissions, source gates, and abstention rules are explicit when relevant.
- Validate JSONL syntax after writing or editing.
- If accepting or rejecting a case, update
.galileo/eval-dataset/changelog.md.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 6d ago First seen · 67 lines · 41 tokens per session scan A 9ced085623e0
eval-dataset is a skill published in the GitHub repository Galileo-Agent-Labs/eval-engineer (41 stars, last pushed 24d ago), licensed MIT. It adds 41 tokens to every session and 614 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
install-gemini-code-review-action
Installs, configures, or upgrades Dazbo's PR code review & issue triage GitHub Action (gemini-review-action) in a target repository. Use when the user wants to set up, configure, or update automated AI code reviews, issue triaging, or Gemini model workflows for their GitHub repository.
convert-to-devto
Converts Markdown content into the format required for Dev.to publication, ensuring correct YAML frontmatter, headers, liquid tags, and inline formatting. Use when the user asks to format or convert an existing blog post or Markdown file for Dev.to.
fw-review
Full Freshworks marketplace app review — iparams, frontend, serverless, FDK, security, and structured text report output — in one skill.
fw-ai-actions-app
Expert-level skill for AI Actions and integrations on Freshworks Platform 3.0. Use when (1) Creating actions.json and SMI functions (flat request, nested response), (2) Request templates and third-party API integration, (3) Pre-build validation (pricing, paywalls, account prerequisites), (4) Failure-case validation…
secrets-with-git-crypt
Manages encryption and decryption of repository secrets (such as .env or .tfvars) using git-crypt, keeping credentials secure with parallel encrypted files. Use when managing secrets, setting up git-crypt, or encrypting/decrypting environment files.
create-md-from-browsermcp-snapshot
Extracts and converts BrowserMCP accessibility tree snapshots into clean, high-fidelity Markdown documents with correct structures (headings, lists, tables, blockquotes, formatting, and links). Use when the user asks to create a markdown document from a webpage; especially if they ask to do it with BrowserMCP.