Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/benchflow-ai/benchflow/task-creatornpx skills add benchflow-ai/benchflow --skill task-creatorgit clone --depth 1 https://github.com/benchflow-ai/benchflowWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00104 | $0.04360 |
| Opus 5 | $0.00052 | $0.02180 |
| Sonnet 5 | $0.00021 | $0.00872 |
| Haiku 4.5 | $0.00010 | $0.00436 |
Grade B, and why
task-creator scanned grade B with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Reads agent configuration directoriesmediumAgent snooping
.claude/, .codex/, .gemini/ hold keys, settings and other credentials a mod has no legitimate need for.
CODEX_MODEL=gpt-5.5 # adjust per ~/.codex/config.toml Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
Bundle frozen inputs (CSVs, xlsx) in `environment/`. For tasks that need internet (research-track), declare the required network and environment policy in `task.md` frontmatter and prefer Playwright over `urllib` for any This is a copy
97% identical to task-creator — 11 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 239 lines — stays where its author put it; the contents beside it link to each section on GitHub.
SkillsBench Task Authoring
Repo context. This skill lives in the
benchflowrepo but authors tasks forbenchflow-ai/skillsbench. Unqualified references below —CONTRIBUTING.md,MAINTAINER.md,docs/*.md,tasks/<task-id>/— mean files in skillsbench, not in this repo. Work from a skillsbench checkout.
Build a task that scores well on the task principles. Two artifacts when you're done: a directory under tasks/<task-id>/ that bench tasks check accepts, and a PR description that maps cleanly to the PR template.
Workflow
1. propose → one-paragraph proposal, gut-check against the proposal rubric
2. scaffold → bench tasks init, plus the native task.md layout below
3. task.md → frontmatter + human-written, outcome-focused prompt body
4. environment → Dockerfile + bundled inputs; do NOT bake skills
5. tests → 4–10 test functions, parametrize for bulk; check formulas AND values
6. oracle → human-written reference solution that derives answers
7. skills → 2–3 generalizable skills (or reuse existing ones from /tasks/*/environment/skills/)
8. validate → bench tasks check + oracle eval (must reach 1.0)
9. self-review → invoke task-review skill on the local path
10. agent runs → Opus 4.8 / latest Codex with and without skills
11. submit → PR with the table the template asks for
Each step is described below. Skip a step only if the rubric says it's optional for your track (research / multimodal). Skipping verification will get the PR rejected.
Step 1 — Propose
Before writing files, write a four-bullet proposal answering the proposal-stage rubric:
- What's the task? One paragraph.
- Who does this in real life? Job, domain, why someone pays for it.
- Why do skills help? What domain knowledge is non-obvious without a skill?
- How would you verify it? Specific output files, deterministic tests.
What ships with it
14 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
- assets/Dockerfile.template 1.2 KB
- assets/task.md.template 834 B
- assets/test.sh.template 915 B
- references/instruction-anatomy.md 3.9 KB
- references/oracle-patterns.md 10 KB
- references/tasktype-code.md 5.0 KB
- references/tasktype-excel.md 5.0 KB
- references/tasktype-infrastructure.md 5.3 KB
- references/tasktype-multimodal.md 5.5 KB
- references/tasktype-research.md 5.0 KB
- references/tasktype-scientific.md 6.3 KB
- references/test-design.md 7.9 KB
- references/time-invariance.md 4.9 KB
- scripts/preflight.sh 2.6 KB runs code
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 239 lines · 104 tokens per session scan B 3785158f65d7
task-creator is a skill published in the GitHub repository benchflow-ai/benchflow (335 stars, last pushed 2d ago), licensed Apache-2.0. It adds 104 tokens to every session and 4,360 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it B with 2 findings (reads agent configuration directories, makes network calls). It is 97% identical to task-creator, differing in 11 lines, and is treated as a copy.
Other skills, from other repositories
lastlight-evals
Scaffold, configure and run a Last Light EVALS workspace — the harness that runs Last Light's real workflows against a mocked GitHub and grades them deterministically. Use when the user wants to "set up / scaffold Last Light Evals", "create an evals workspace or instance", "run evals", "compare models", or author new…
hotpath_init
Configure hotpath profiling in a Rust project. Adds the hotpath dependency with feature-gated setup, instruments main with hotpath::main, functions with measure/measureall, and wraps channels, mutexes, rwlocks, streams, futures, reqwest clients, axum routers and byte-level I/O with hotpath macros. Use when the user…
review-pr
Walk Sean through an incoming waku-agent PR or issue and present it his way — four fixed sections: what this is, why it matters, how HE can test it with you as copilot, and are we ready to merge / reply / close and why. Use whenever Sean asks to look at, test, triage, or decide on a pull request or an issue, and…
compose-graphics
Advanced Compose visuals - Material 3 Expressive motion physics, AGSL shaders (Android 13+), Canvas/DrawScope generative, graphicsLayer effects.
canvas-generative
Algorithmic and generative art with Canvas 2D - particles, flow fields, noise, fractals, L-systems.
writing-bench-task-judge
Use when writing or modifying checkgoals() / getanswer() / App check methods in benchenv/task/, or when reviewing a draft task's judge correctness. Triggers include adding a new task, editing a judge method, or diagnosing a judge false-positive/negative.