A generator of ADE-Bench benchmark tasks from a dbt project. dbt is a tool for building and testing data models with SQL, while a benchmark task tests whether an AI agent can solve a realistic problem.

Scanning dbt models and producing task files, setup and solution scripts, patches, and SQL tests for different difficulty levels.

It turns an existing working project into repeatable tests by creating controlled bugs and checking whether the agent repairs them correctly.

create-task — Skill for Claude Code by typedef-ai/ade-bench-plugin

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add typedef-ai/ade-bench-plugin --skill create-task

Clone the repo

git clone --depth 1 https://github.com/typedef-ai/ade-bench-plugin

Made for: Claude Code.

Or install ade-bench, the plugin that ships this one along with the rest of its 3 skills, 3 commands, 1 agent.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/typedef-ai/ade-bench-plugin/create-task/github.svg)](https://agentmods.dev/skills/typedef-ai/ade-bench-plugin/create-task)

Your own site

<a href="https://agentmods.dev/skills/typedef-ai/ade-bench-plugin/create-task"><img src="https://agentmods.dev/badge/skills/typedef-ai/ade-bench-plugin/create-task/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/typedef-ai/ade-bench-plugin/create-task"><img src="https://agentmods.dev/badge/skills/typedef-ai/ade-bench-plugin/create-task.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 66 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 7,169 The whole file, excluding the scripts and references it only reads on demand.

Security scan B 1 finding. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00066	$0.07169
Opus 5.5	$0.00026	$0.02868
Sonnet 5.5	$0.00013	$0.01434
Haiku 4.5	$0.00007	$0.00717

Measured 2mo ago against content hash 8c50118e7e72, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade B, and why

create-task scanned grade B with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Recursive force deletemediumDestructive command

rm -rf with a variable or a broad path is one typo away from removing the wrong tree.

rm -rf /tmp/ade-bench-validate-{task_id}

Downgraded: this mod is about security review, or the phrase is quoted, so it is likely naming the pattern rather than instructing it.

skills/create-task/SKILL.md · 616 lines

How it starts

The opening of the file, as written. The whole thing — 616 lines — stays where its author put it; the contents beside it link to each section on GitHub.

ADE-Bench Task Generator

You are a benchmark task generator for ADE-Bench, a framework that evaluates AI agents on dbt and SQL tasks. Your job is to analyze a user's dbt project, propose realistic bug-injection tasks, and generate the full scaffolding needed to run those tasks as benchmarks.

Overview

ADE-Bench tasks work by:

Taking a working dbt project
Introducing a bug via a patch (setup)
Giving an agent a prompt describing the symptom
Evaluating whether the agent fixes the bug (via dbt tests + table comparison)

You will generate tasks that follow this pattern using the user's own dbt project and data.

Phase 1: Parse Arguments and Discover the Project

The user invokes this skill with a path to their dbt project. Parse the arguments:

First positional arg: Path to the dbt project directory (required). This directory must contain dbt_project.yml.
--output-dir: Where to write generated tasks (default: ./ade-bench-tasks next to the project).
--no-copy: If present, reference the original project in-place instead of copying it. Default: copy into {output_dir}/projects/.
--db-type: Database type override (duckdb or snowflake). Default: auto-detected from profiles.yml.
--db-name: Name of the database (without extension for DuckDB, or the Snowflake database name). Default: inferred from profiles.yml.
--db-path: Path to the directory containing the DuckDB file (DuckDB only). When provided, this value is written to db_dir in task.yaml. When omitted, do NOT write db_dir at all — the harness falls back to ~/.ade-bench/shared/databases/duckdb/<db_name>.duckdb, which is correct for any bundled ade-bench project (airbnb, f1, quickbooks, etc.).

Verify ade-bench is Installed

Before discovery, check whether ade-bench (the harness that runs the generated tasks) is available. Generation works without it, but the user can't run the output without it.

Detection — in order:

command -v ade — if it returns a path and ade --help succeeds, ade-bench is installed.
test -f ~/.ade-bench/pyproject.toml && grep -q 'name = "ade-bench"' ~/.ade-bench/pyproject.toml — fallback if ~/.local/bin isn't on PATH yet.

Read the full file on GitHub · 616 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

2mo ago First seen · 616 lines · 66 tokens per session scan B 8c50118e7e72

Subscribe to this mod's changes

create-task is a skill published in the GitHub repository typedef-ai/ade-bench-plugin (3 stars, last pushed 6mo ago), licensed MIT. It adds 66 tokens to every session and 7,169 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it B with 1 finding (recursive force delete). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Claude Code Data and AI Testing ai-agents analytics-engineering benchmark claude-code-plugin dbt

create-task next to 6 skills from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
create-tasktypedef-ai/ade-bench-plugin	Skill	3	6mo ago	MIT	B	66	Claude Code
authoring-meta-contextkeithbinkly/dbt-meta-context	Skill	21	1mo ago	MIT	A	97	Claude Code, Codex
dbt-agent-readinessGetCassis/dbt-agent-readiness	Skill	11	2mo ago	MIT	A	126	Claude Code
add-ai-webapimicrosoft/power-platform-skills	Skill	1.1k	yesterday	MIT	A	226	Claude Code
openai-visionbenchflow-ai/skillsbench	Skill	1.9k	3d ago	Apache-2.0	A	17	Claude Code, Codex
prefix-cache-replaybenchflow-ai/skillsbench	Skill	1.9k	3d ago	Apache-2.0	A	107	Claude Code, Codex
rl-post-trainingbenchflow-ai/skillsbench	Skill	1.9k	3d ago	Apache-2.0	A	126	Claude Code, Codex

create-task