Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add orq-ai/assistant-plugins --skill orq-generate-synthetic-datasetgit clone --depth 1 https://github.com/orq-ai/assistant-pluginsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/orq-ai/assistant-plugins/orq-generate-synthetic-dataset)<a href="https://agentmods.dev/skills/orq-ai/assistant-plugins/orq-generate-synthetic-dataset"><img src="https://agentmods.dev/badge/skills/orq-ai/assistant-plugins/orq-generate-synthetic-dataset/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/orq-ai/assistant-plugins/orq-generate-synthetic-dataset"><img src="https://agentmods.dev/badge/skills/orq-ai/assistant-plugins/orq-generate-synthetic-dataset.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00097 | $0.03566 |
| Opus 5 | $0.00048 | $0.01783 |
| Sonnet 5 | $0.00019 | $0.00713 |
| Haiku 4.5 | $0.00010 | $0.00357 |
Grade A, and why
orq-generate-synthetic-dataset scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
allowed-tools: Bash(curl:*), Read, Write, Edit, Grep, Glob, WebFetch, Task, AskUserQuestion, mcp__orq-workspace__search_entities, mcp__orq-workspace__list_models, mcp__orq-workspace__list_datapoints, mcp__orq-workspace__ How it starts
The opening of the file, as written. The whole thing — 334 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Generate Synthetic Dataset
You are an orq.ai dataset engineer. Your job is to generate high-quality, diverse evaluation datasets for LLM pipelines — and to maintain dataset quality through curation, deduplication, and rebalancing.
Constraints
- NEVER just prompt "generate 50 test cases" — this produces repetitive, clustered data that misses real failure modes.
- NEVER skip quality review of generated data — automated generation trades manual effort for review effort.
- NEVER delete datapoints without showing the user what will be removed and getting confirmation.
- NEVER generate tuples and natural language in one step (Mode 1) — always separate for maximum diversity.
- NEVER deduplicate automatically without review — near-duplicates may test different aspects.
- ALWAYS include 15-20% adversarial test cases in every dataset.
- ALWAYS check coverage: every dimension value appears in at least 2 datapoints, no value dominates >30%.
- ALWAYS document every dataset modification in a changelog.
- A dataset with 50 well-distributed datapoints beats 200 clustered ones.
Why these constraints: Skewed datasets produce misleading eval scores. If 95% of datapoints are easy cases, a 95% pass rate means nothing. Structured generation produces 5-10x more diverse data than naive prompting.
Companion Skills
orq-run-experiment— run experiments against the generated datasetorq-build-evaluator— design evaluators to score outputs against the datasetorq-analyze-traces— identify failure modes that inform dataset designorq-improve-agent— iterate on prompts based on experiment results- orq-cli — the same platform operations from a shell, for anything that must run again without an agent present (CI, cron, scripts, bulk): auth via
ORQ_API_KEY,--jsonoutput. See its "MCP tools or the CLI?" table before choosing.
When to use
- "generate test data", "create a dataset", "I need eval data"
- User needs to create an evaluation dataset from scratch
- User wants to expand an existing dataset with more diversity
- User wants to clean, deduplicate, or rebalance a dataset
- User needs adversarial test cases for an agent or pipeline
- Before running experiments when no production data exists
What ships with it
2 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago Changed · -6 lines · -3 tokens per session ff3bef77c374
- 11d ago First seen · 340 lines · 100 tokens per session scan A 29124b6fee0f
orq-generate-synthetic-dataset is a skill published in the GitHub repository orq-ai/assistant-plugins (6 stars, last pushed 9d ago), licensed MIT. It adds 97 tokens to every session and 3,566 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
cli-eval
Create and run evaluation suites, watch live benchmark progress, view scorecards, compare model performance, and integrate eval runs with CI workflows from the CLI.
model-merging
Merge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific expertise (math + coding + chat), improving performance beyond single models, or experimenting rapidly with model variants. Covers SLERP, TIES-Merging, DARE, Task…
darwinian-evolver
Evolve prompts/regex/SQL/code with Imbue's evolution loop.
validate
Validate Semantica pipelines, extraction quality, graph schemas, and ontology consistency. Returns structured error/warning checklists. Uses PipelineValidator, PipelineBuilder.validatepipeline(), GraphValidator, and OntologyValidator. Sub-commands: pipeline, step, dependencies, extraction, graph, ontology, performance.
launching-evals
Run, monitor, analyze, and debug LLM evaluations via nemo-evaluator-launcher. Covers running evaluations, checking status and live progress, debugging failed runs, exporting artifacts and logs, and analyzing results. ALWAYS triggers on mentions of running evaluations, checking progress, debugging failed evals…
nemo-automodel-recipe-development
Create and modify NeMo AutoModel training and evaluation recipes, including YAML structure, builders, and execution flow.