ax-task-design

A method for designing small, repeatable coding-agent tasks that measure the developer experience of different tool interfaces.

In plain words
What is it for?
Use it to create benchmark tasks for raw documentation, command-line tools, Skills, MCP servers, or plugins, with restart checks, fixture records, repetitions, and clear stopping rules.
Why use it?
It makes results easier to reproduce and compare by fixing the starting files, allowed side effects, verifier, task limits, and source of truth.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/shenli/devtool-ax-kit/ax-task-design
Any agent
npx skills add shenli/devtool-ax-kit --skill ax-task-design
Clone the repo
git clone --depth 1 https://github.com/shenli/devtool-ax-kit

Made for: Claude Code, Codex.

Per session 16 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 181 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00016 $0.00181
Opus 5 $0.00008 $0.00090
Sonnet 5 $0.00003 $0.00036
Haiku 4.5 $0.00002 $0.00018

Measured 2d ago against content hash 93b1539959d9, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

ax-task-design scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/ax-task-design/SKILL.md · 20 lines

What it actually says

Create a small, realistic task with a frozen starting fixture, explicit allowed side effects, an independent verifier, and a clear stop condition. Separate the task prompt from the verifier. Define what the agent may use (raw docs, CLI, Skill, MCP, or plugin) so surface comparisons are attributable. Prefer disposable local or test resources; never require production credentials.

Make the task legible from the agent's available context: state where the source of truth, logs, acceptance criteria, and recovery instructions live. Include at least one boundary or restart assertion when the task has durable state.

Record the fixture hash, prompt, harness/model, repetitions, and human interventions. Keep the task narrow enough that a failure can be reproduced and reported to the tool owner.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 20 lines · 16 tokens per session scan A 93b1539959d9

Subscribe to this mod's changes

ax-task-design is a skill published in the GitHub repository shenli/devtool-ax-kit (2 stars, last pushed 3d ago), licensed Apache-2.0. It adds 16 tokens to every session and 181 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

copilotkit-setup

Use when adding CopilotKit to an existing project or bootstrapping a new CopilotKit project from scratch. Covers framework detection, package installation, runtime wiring (managed Intelligence or self-hosted SSE), provider setup, and first working chat integration.

CopilotKit/CopilotKit · 56 tokens

a2ui-renderer

Render A2UI (Agent-to-UI declarative surfaces) in CopilotKit v2. Enable the runtime via CopilotRuntime({ a2ui: {...} }), then enable the provider via . Auto-activates via /info — do NOT manually pass renderActivityMessages. createA2UIMessageRenderer ships from @copilotkit/react-core/v2; low-level primitives…

CopilotKit/CopilotKit · 175 tokens

copilotkit-develop

Use when building AI-powered features with CopilotKit v2 -- adding chat interfaces, registering frontend tools, sharing application context with agents, handling agent interrupts, and working with the CopilotKit runtime.

CopilotKit/CopilotKit · 46 tokens

copilotkit-upgrade

Use when migrating a CopilotKit v1 application to v2 -- updating package imports, replacing deprecated hooks and components, switching from GraphQL runtime to AG-UI protocol runtime, and resolving breaking API changes.

CopilotKit/CopilotKit · 48 tokens

copilotkit-debug

Use when diagnosing CopilotKit issues -- runtime connectivity failures, agent not responding, streaming errors, tool execution problems, transcription failures, version mismatches, and AG-UI event tracing.

CopilotKit/CopilotKit · 42 tokens

react-core

@copilotkit/react-core — mount the CopilotKit provider (from @copilotkit/react-core/v2) in a Next.js App Router / React Router v7 / TanStack Start / SPA app, drop in CopilotChat/CopilotPopup/CopilotSidebar (v2 chat components ship from react-core/v2 — NOT react-ui, which is CSS-only in v2), access and subscribe to…

CopilotKit/CopilotKit · 191 tokens