Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/frontisai/naturebench/task-verifynpx skills add FrontisAI/NatureBench --skill task-verifygit clone --depth 1 https://github.com/FrontisAI/NatureBenchWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00038 | $0.01273 |
| Opus 5 | $0.00019 | $0.00636 |
| Sonnet 5 | $0.00008 | $0.00255 |
| Haiku 4.5 | $0.00004 | $0.00127 |
Grade A, and why
task-verify scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 119 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Task-Verify Skill
Verify that a task package built by task-build is valid, consistent, and usable as a benchmark.
Input Requirements
User provides:
- task_package_path: Path to task package directory containing problem/, evaluation/, metadata.json
Optional:
- filter_result.json: If present in task package, used for algorithm name verification (C2.3, C2.4)
Verification Workflow
Execute 5 phases with 36 checks total. If Phase 0 fails, terminate immediately. The check count MUST be exactly 36. Do not skip, merge, or invent checks beyond this list.
Phase 0: File Completeness & Structure (5 checks)
Fast-fail validation. See references/verification_rules.md for detailed rules.
- C0.1: Required files exist
- C0.2: Instance directories match between problem/data/ and evaluation/ground_truth/
- C0.3: metadata.json is valid JSON with required fields
- C0.4: evaluator.py has valid Python syntax
- C0.5: Dockerfile.v3 dependency viability (compatibility, availability, API compatibility)
Phase 1: Cross-File Consistency (9 checks)
Ensure all documents agree on key information. See references/verification_rules.md.
- C1.1: Task name matches between README and metadata.json
- C1.2: Instance names consistent across all files and directories
- C1.3: Metric names consistent across README, metadata, evaluator
- C1.4: Metric directions match between README and metadata
- C1.5: Exactly one primary metric designated in README
- C1.6: Output format in README matches evaluator expectations
- C1.7: Ground truth paths in evaluator actually exist
- C1.8: data_description.md directory structure matches actual files in problem/data/
- C1.9: data_description.md has required section structure (Directory Structure, Dataset Overview, File Formats & Schemas, Special Notes)
Phase 2: Information Firewall Verification (6 checks)
Prevent paper/algorithm information leakage. See references/verification_rules.md.
What ships with it
4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 3d ago First seen · 119 lines · 38 tokens per session scan A c6a659440fa1
task-verify is a skill published in the GitHub repository FrontisAI/NatureBench (106 stars, last pushed 3d ago), licensed MIT. It adds 38 tokens to every session and 1,273 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
systematic-debugging
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes.
brainstorming
You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. Explores user intent, requirements and design before implementation.
auto-perf-optimize
Run agent-driven VS Code performance or memory investigations. Use when asked to launch Code OSS, automate a VS Code scenario, run the Chat memory smoke runner, capture renderer heap snapshots, take workflow screenshots, compare run summaries, or drive a repeatable scenario before heap-snapshot analysis.
chat-perf
Run chat perf benchmarks and memory leak checks against the local dev build or any published VS Code version. Use when investigating chat rendering regressions, validating perf-sensitive changes to chat UI, or checking for memory leaks in the chat response pipeline.
chat-pet-sprite-creation
Use when creating or changing VS Code chat pet sprite art, sprite sheets, state animations, eye treatments, Stable/Insiders variants, or pet transitions under src/vs/workbench/contrib/chat/browser/widget/media/chatPet.
cpu-profile-analysis
Analyze V8/Chrome CPU profiles (.cpuprofile) and DevTools trace files (Trace-.json). Use when: profiling performance, investigating slow functions, comparing code paths, finding bottlenecks, analyzing timeToRequest, understanding call trees from sampling profiler data, analyzing layout/paint/rendering, investigating…