Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add instructions/r0b0tlab/r0b0bench/agents-mdgit clone --depth 1 https://github.com/r0b0tlab/r0b0benchWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/instructions/r0b0tlab/r0b0bench/agents-md)<a href="https://agentmods.dev/instructions/r0b0tlab/r0b0bench/agents-md"><img src="https://agentmods.dev/badge/instructions/r0b0tlab/r0b0bench/agents-md.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00903 | $0.00903 |
| Opus 5 | $0.00451 | $0.00451 |
| Sonnet 5 | $0.00181 | $0.00181 |
| Haiku 4.5 | $0.00090 | $0.00090 |
Grade A, and why
r0b0bench AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 60 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AGENTS.md — working on / running r0b0bench
This file is for coding agents (Hermes, Cursor, etc.).
What this repo is
- Runnable OpenAI-compatible benchmark client (
r0b0benchCLI + optional container). - Not a model server. Do not bake weights into this image.
- Exactly four public profiles:
core|core-subset|systems|hard-subset. The first two always include the systems block;systemsruns only that block;hard-subsetis the hard-multiturn quality profile (MultiChallenge + BFCL-MT with canary gates at both ends, no full systems block; τ²-bench lanes removed 2026-08-22 — no value added).
Systems block (every profile)
-
Order:
canary→bfcl_mt→bfcl_ast→latency→concurrency→throughput→niah -
NIAH depths = 25% / 50% / 90% of
(max_model_len − 64)from/v1/models. Do not hardcode 8k/16k/28k. -
Latency/concurrency/throughput use the portable OpenAI chat backend (label method strings; never mix with vllm-bench rows).
Chat-template control (runtime variable)
Lanes never set chat_template_kwargs. The only channel is the runtime env
R0B0BENCH_CHAT_TEMPLATE_KWARGS (JSON object), merged into every chat request
by endpoint._chat_body and into the BFCL adapters via extra_body. With the
variable unset, no template kwargs are sent and the model's served template
default applies (e.g. thinking ON for thinking-default models). Record the
exact value used in every run manifest; rows measured under different template
states are separate ledger entries.
Hard rules
- Output outside checkout —
--outputmust not be inside the git tree (except gitignoredout/). Private evidence must not be committed. - No BFCL forks — use official
bfcl-evalcategories/names; do not invent leaderboard-looking composites. - Canary taxonomy — stop the run on infra/auth/schema death; ordinary wrong answers are scored (canary content fails → FAIL, not silent continue for publish).
- No score washing — do not weaken verifiers to make a model pass.
--skip-systemsis debug-only; report must setinvalid_for_publish=true.- Do not claim r0b0bench results without a package-produced
report.json. - Secrets only via env; never print tokens.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 60 lines · 903 tokens per session scan A 703db14aa8d3
r0b0bench AGENTS.md is an instructions file published in the GitHub repository r0b0tlab/r0b0bench (11 stars, last pushed 14d ago), licensed MIT. It adds 903 tokens to every session, about $0.0045 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other instructions, from other repositories
bernstein AGENTS.md
AGENTS.md instructions for sipyourdrink-ltd/bernstein, covering bernstein - agents.md, module map, directory context files, build & test and setup.
bernstein copilot-instructions.md
Copilot instructions for sipyourdrink-ltd/bernstein, covering copilot instructions, architecture, key constraints, before committing and git rules.
benchmark-radar AGENTS.md
AGENTS.md instructions for ktwu01/benchmark-radar, covering repository instructions, glob rule: showcase and ui communication, example: simplify badge copy and keep its style, glob rule: benchmark radar audience and pull request merges.
flameox AGENTS.md
AGENTS.md instructions for morluto/flameox, covering repository guidelines, product direction, project structure & module organization, build, test, and development commands and coding style & naming conventions.
relarena AGENTS.md
AGENTS.md instructions for PriorLabs/relarena, covering relarena — agent notes, this repo is public, adding or changing a model, refactor-friendly structure rules and macos: run tests / cli with ompnumthreads=1.
AutoRAG AGENTS.md
AGENTS.md instructions for Marker-Inc-Korea/AutoRAG, covering autorag — pi-powered librarian agent, git workflow (binding), before any pr review, pr work, or new feature, after the work is finished and developer commands.