Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add instructions/philolabs/agentic-vbench/agents-mdgit clone --depth 1 https://github.com/PhiloLabs/agentic-vbenchWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00887 | $0.00887 |
| Opus 5 | $0.00443 | $0.00443 |
| Sonnet 5 | $0.00177 | $0.00177 |
| Haiku 4.5 | $0.00089 | $0.00089 |
Grade A, and why
agentic-vbench AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 54 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AGENTS.md
Telegraph style. Root rules only. Read scoped AGENTS.md before subtree work.
Start
- Repo:
agentic-vbench. Greenfield; conventions still forming — keep changes minimal and reversible. - Replies: repo-root refs only, e.g.
src/foo.ts:80. No absolute paths, no~/. - Verify before deciding. Read source/tests/current behavior; do not assume APIs, defaults, or runtime from prior knowledge.
- This file is policy;
docs/(when added) is explanation; code/config is the contract. Link across layers, never duplicate. - Add new conventions/workflows here (or in a scoped
AGENTS.md) — not in a code comment or commit message. - New
AGENTS.mdin any subtree: add a siblingCLAUDE.mdsymlink (ln -s AGENTS.md CLAUDE.md).
Map
tasks/— Harbor task dirs, grouped into 4 family subdirs:tasks/agentic_vbench_repair/— 18exp-*audio/video repair tasks (audio / color / deblur / sr / swap / glitch / cut / disfluency).tasks/agentic_vbench_assembly/— 18 video-assembly tasks (agentic-vbench-assembly-task<N>).tasks/agentic_vbench_sequencing/— 28 video-ordering tasks (agentic-vbench-sequencing-task<N>).tasks/agentic_vbench_repurpose/— 36 long-form-to-short-vertical repurpose tasks (per-task creative brief + LLM-judged rubric). Downstream tools resolve task names to paths viascripts/_task_paths.py(task_dir(name),all_tasks(family=…)), so adding/renaming families means touching one file.
scripts/— the Harbor wrapper.install-harbor.shpins the Harbor CLI;parallel_rollout.pyschedulesharbor runacross many tasks against either local Docker or Modal;monitor_job.pytails a running trial;_task_paths.pyis the name→path resolver. Tasks are evaluated using the verifier code shipped under each task'ssteps/solve/tests/; Harbor produces the per-trialreward.json.docs/VERIFIER_DESIGN.md— load-bearing doc for the verifier math (universal normalize-improvement).jobs/,site/,logs/,experiments/— runtime / local-only outputs (gitignored).- Scoped
AGENTS.mdfiles: none yet. Add one when a subtree has rules that genuinely differ from root.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 3d ago First seen · 54 lines · 887 tokens per session scan A 62f6fde8a9fc
agentic-vbench AGENTS.md is an instructions file published in the GitHub repository PhiloLabs/agentic-vbench (89 stars, last pushed 4d ago), licensed Apache-2.0. It adds 887 tokens to every session, about $0.0044 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other instructions, from other repositories
benchjack CLAUDE.md
Instructions for benchjack/benchjack, covering claude.md, project overview, tech stack, repository layout and common commands.
finreportbench AGENTS.md
Instructions for MisterBrookT/finreportbench, covering finreportbench agent handoff, mission, start here, repository map and dataset contract.
mcpscope AGENTS.md
Instructions for ffleurey/mcpscope, covering mcpscope agent guide, fast start, project shape, working style and inspect a running mcpscope with mcpscope.
prompty copilot-instructions.md
Instructions for microsoft/prompty, covering prompty v2 — complete rebuild plan, what prompty uses from the typra-generated model, key design decisions, python coding rules and environment & tooling.
mobilegym AGENTS.md
Instructions for Purewhiter/mobilegym, covering agents.md, project overview, type-checking strategy, eslint and with data graph generation.
benchflow AGENTS.md
Instructions for benchflow-ai/benchflow, covering benchflow, setup + test, conventions and skill catalog (.agents/skills, mirrored at .claude/skills).