Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add growthxai/output --skill output-eval-dataset-designgit clone --depth 1 https://github.com/growthxai/outputWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/growthxai/output/output-eval-dataset-design)<a href="https://agentmods.dev/skills/growthxai/output/output-eval-dataset-design"><img src="https://agentmods.dev/badge/skills/growthxai/output/output-eval-dataset-design/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/growthxai/output/output-eval-dataset-design"><img src="https://agentmods.dev/badge/skills/growthxai/output/output-eval-dataset-design.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 7 findings, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium MCP Rug Pull · line 138 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 142 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 181 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 232 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 275 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 278 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 281 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00038 | $0.02610 |
| Opus 5 | $0.00019 | $0.01305 |
| Sonnet 5 | $0.00008 | $0.00522 |
| Haiku 4.5 | $0.00004 | $0.00261 |
Grade A, and why
output-eval-dataset-design scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 300 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Designing Eval Datasets
Overview
Diverse datasets catch more failures. This skill teaches dimension-based dataset design — systematically varying inputs along axes that target failure-prone regions of your workflow. The output is a set of YAML dataset files ready for output workflow test.
When to Use
- Bootstrapping an eval dataset for a new workflow
- Existing datasets only cover happy paths
- Real production traces are sparse (fewer than 50)
- Stress-testing specific failure hypotheses from error analysis
When NOT to Use
- You already have 100+ representative real traces — use stratified sampling from those instead of generating synthetic data
- You haven't done error analysis yet — do that first (
output-eval-error-analysis) so your dimensions target real failure modes, not guesses
Step 1: Define Dimensions
Identify 3+ axes of input variation that target anticipated failure modes. Each dimension should vary one aspect of the input that you expect to influence output quality.
Deriving dimensions from error analysis
Map failure categories to input properties that trigger them:
| Failure Category | Triggering Input Property | Dimension |
|---|---|---|
| Off-topic drift | Ambiguous or broad topics | Topic specificity: specific / broad / ambiguous |
| Tone mismatch | Conflicting tone signals | Tone difficulty: simple / nuanced / contradictory |
| Too short | Short or vague input | Input detail: minimal / moderate / comprehensive |
| Missing sections | Many explicit requirements | Requirement count: 0 / 1-2 / 5+ |
| Hallucinated URLs | Technical topics with real entities | Entity density: none / few / many |
Example dimensions for a blog generation workflow
| Dimension | Values | Why |
|---|---|---|
| Topic complexity | simple, technical, ambiguous | Technical and ambiguous topics trigger more hallucination and drift |
| Tone request | none, formal, casual, contradictory | Explicit tone requests reveal tone-matching failures |
| Length constraint | none, short (100w), long (1000w) | Extreme length constraints trigger truncation and padding |
| Required sections | none, 1 section, 3+ sections | Multiple required sections stress structural compliance |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 10d ago First seen · 300 lines · 38 tokens per session scan A eb77d7756480
output-eval-dataset-design is a skill published in the GitHub repository growthxai/output (435 stars, last pushed today), licensed Apache-2.0. It adds 38 tokens to every session and 2,610 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
ax-agent-rlm
This skill helps an LLM generate correct AxAgent RLM/runtime code using @ax-llm/ax. Use when the user asks about RLM code execution, AxJSRuntime, contextFields, contextPolicy, liveRuntimeState, promptLevel, stage prompt controls, executorModelPolicy, maxRuntimeChars, agent.test(...), llmQuery(...), recursionOptions…
ax-ai
This skill helps an LLM generate correct AI provider setup and configuration code using @ax-llm/ax. Use when the user asks about ai(), providers, models, routing, adaptive balancing, presets, embeddings, batch audio with ai.transcribe() or ai.speak(), extended thinking, context caching, or mentions…
ax-flow
This skill helps an LLM generate correct AxFlow workflow code using @ax-llm/ax. Use when the user asks about flow(), AxFlow, workflow orchestration, parallel execution, DAG workflows, conditional routing, map/reduce patterns, or multi-node AI pipelines.
ax-gen
This skill helps an LLM generate correct AxGen code using @ax-llm/ax. Use when the user asks about ax(), AxGen, generators, forward(), streamingForward(), validation, assertions, streaming assertions, field processors, step hooks, self-tuning, or structured outputs. For MCP clients, transports, prompts, resources…
ax-signature
This skill helps an LLM generate correct DSPy signature code using @ax-llm/ax. Use when the user asks about signatures, s(), f(), field types, string syntax, fluent builder API, validation constraints, or type-safe inputs/outputs.
ax-cpp-ai
Use when writing C++ code with axllm for named deployment profiles, generic provider clients, model selection, OpenAI-compatible calls, Responses, Gemini, Anthropic, routers, and balancers.