setup

A setup skill for installing or checking the ADE-Bench harness, the separate Python project that runs generated benchmark tasks in Docker.

In plain words
What is it for?
It is for verifying or installing Git, uv, and Docker prerequisites, cloning the harness, installing its dependencies, and placing the command on the system path.
Why use it?
It checks whether the tools needed to execute benchmarks are available and makes the harness accessible through the ade command.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/typedef-ai/ade-bench-plugin/setup
Any agent
npx skills add typedef-ai/ade-bench-plugin --skill setup
Clone the repo
git clone --depth 1 https://github.com/typedef-ai/ade-bench-plugin

Made for: Claude Code, Codex.

Per session 79 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 2,544 The whole file, excluding the scripts and references it only reads on demand.
Security scan C 2 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00079 $0.02544
Opus 5 $0.00039 $0.01272
Sonnet 5 $0.00016 $0.00509
Haiku 4.5 $0.00008 $0.00254

Measured 2d ago against content hash 4ee2ebfbad05, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade C, and why

setup scanned grade C with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Downloads and executes remote codehighSupply chain

curl | sh runs whatever the server returns today, which is not necessarily what it returned when this was reviewed.

- **uv**: `curl -LsSf https://astral.sh/uv/install.sh | sh` (then restart the shell or `source ~/.zshrc`).

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

- **uv**: `curl -LsSf https://astral.sh/uv/install.sh | sh` (then restart the shell or `source ~/.zshrc`).
skills/setup/SKILL.md · 227 lines

How it starts

The opening of the file, as written. The whole thing — 227 lines — stays where its author put it; the contents beside it link to each section on GitHub.

ADE-Bench Setup

The plugin generates task definitions. ade-bench (a separate Python project) is what actually executes them inside Docker against the user's data. This skill makes ade-bench available so generated tasks can be run.

Default install location: ~/.ade-bench. The skill git clones ade-bench there, runs uv sync to install Python deps, and uv tool install . to put the ade CLI on PATH.


Step 1: Parse Arguments

  • --path <install-path>: Where to install ade-bench. Default: ~/.ade-bench.

Resolve ~ to the user's home directory. Use the resolved absolute path in every step below.


Step 2: Check Prerequisites

Run these in parallel and collect the results:

  1. git: command -v git
  2. uv: command -v uv
  3. docker: command -v docker

If any of the three is missing, stop and tell the user what's missing and how to install it:

  • git: Use the system package manager (brew install git, apt install git, etc.).
  • uv: curl -LsSf https://astral.sh/uv/install.sh | sh (then restart the shell or source ~/.zshrc).
  • docker: Recommend Docker Desktop, Colima, or OrbStack. Don't try to install or start Docker. The user must do this.

If docker is installed but not running, that's fine for setup — flag it but proceed. Docker only needs to be running when tasks actually execute.


Step 3: Detect Existing Installation

Two checks, in order:

  1. Already installed via this skill?
    test -f <install-path>/pyproject.toml && grep -q 'name = "ade-bench"' <install-path>/pyproject.toml
    
  2. Already on PATH from somewhere else?
    command -v ade
    

If either succeeds:

  • Run ade --help to confirm the CLI works.
  • Tell the user where ade-bench was found and that no install is needed.
  • Skip to Step 7 (DuckDB database download — still verify it's been done) and continue from there.

If <install-path> exists but is not a valid ade-bench checkout (different repo, partial clone, leftover files), stop and ask the user whether to remove and reinstall or pick a different path. Do not silently overwrite.

Read the full file on GitHub · 227 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 227 lines · 79 tokens per session scan C 4ee2ebfbad05

Subscribe to this mod's changes

setup is a skill published in the GitHub repository typedef-ai/ade-bench-plugin (3 stars, last pushed 3mo ago), licensed MIT. It adds 79 tokens to every session and 2,544 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it C with 2 findings (downloads and executes remote code, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

pr-verify

Verify a Docglow change actually works before submitting or merging a PR. Runs the conformance suite, then a behavioral verification pass (flag matrix, artifact-join spot checks, pipeline contract sweep, payload budget). Use when reviewing a PR, self-reviewing a branch before opening a PR, or when asked to "verify…

docglow/docglow · 79 tokens

dbt-agent-readiness

Audit a dbt project for agent-readiness: what would an AI agent get wrong if you pointed it at this data today? Produces a prioritized report organized by failure modes (wrong numbers, wrong table, wrong column, can't join, query fails). Scales via two-pass architecture with parallel subagents. Each subagent reads its…

GetCassis/dbt-agent-readiness · 126 tokens

authoring-meta-context

Authors and extracts dbt MetricFlow meta context — structured YAML meta: blocks that encode business knowledge (thresholds, investigation paths, SLAs, relationships) alongside metric definitions so AI agents answer analytical questions accurately. Use when user asks about context cards, meta context blocks, dbt metric…

keithbinkly/dbt-meta-context · 97 tokens

animation-best-practices

CSS and UI animation patterns for responsive, polished interfaces. Use when implementing hover effects, tooltips, button feedback, transitions, or fixing animation issues like flicker and shakiness.

northgraindata/dbt-doctor · 42 tokens

dbt-doctor

Static analysis and health checks for dbt projects. Use before committing SQL/YAML or when enforcing CI quality gates.

northgraindata/dbt-doctor · 28 tokens

dbt-expert

Expert-level dbt (data build tool), models, tests, documentation, incremental models, macros, and Jinja templating.

personamanagmentlayer/pcl · 30 tokens