Getting it into your agent
This one installs as part of its plugin. Adding the marketplace and installing the plugin brings it with everything else the plugin ships.
/plugin marketplace add vibeic/vibe-ic/plugin install vibe-icWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/commands/vibeic/vibe-ic/vibe-ic-benchmark)<a href="https://agentmods.dev/commands/vibeic/vibe-ic/vibe-ic-benchmark"><img src="https://agentmods.dev/badge/commands/vibeic/vibe-ic/vibe-ic-benchmark/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/commands/vibeic/vibe-ic/vibe-ic-benchmark"><img src="https://agentmods.dev/badge/commands/vibeic/vibe-ic/vibe-ic-benchmark.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00086 | $0.01348 |
| Opus 5 | $0.00043 | $0.00674 |
| Sonnet 5 | $0.00017 | $0.00270 |
| Haiku 4.5 | $0.00009 | $0.00135 |
Grade A, and why
vibe-ic-benchmark scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 84 lines — stays where its author put it; the contents beside it link to each section on GitHub.
/vibe-ic-benchmark — turnkey benchmark runner
This command is the front door for every open IC-design benchmark. It enforces
the methodology from open-benchmark-methodology skill (§ 2 decision matrix → § 3
substitution disclosure → § 4 triage rubric) by routing to the correct run-shape
per the registry at ${CLAUDE_PLUGIN_ROOT}/benchmark/BENCHMARK_REGISTRY.json.
--solve names the complete route-and-solve lifecycle; it is not a direct
solver entry. For every problem, the first decision is made by the general
task_nature_route from the visible prompt and supplied-RTL state. Only after
that route selects the normal flow entry/evidence boundary may
vibe_ic_one_shot_runner run. Benchmark name, problem id, and dataset metadata
never select a route.
Modes
# 1. List all known benchmarks + their shape + status
python3 ${CLAUDE_PLUGIN_ROOT}/programs/benchmark_dispatch.py --list
# 2. Show plan for one benchmark (env check + recommended commands)
python3 ${CLAUDE_PLUGIN_ROOT}/programs/benchmark_dispatch.py <bench>
# 3. Run Program First through the general flow; this creates a fresh run dir
# and emits AI backup/review worklists
python3 ${CLAUDE_PLUGIN_ROOT}/programs/benchmark_dispatch.py <bench> \
--solve --dataset <path-to-dataset> --run <run-dir>
# 4. Complete blind AI review; a FAIL also needs an executable challenge
python3 ${CLAUDE_PLUGIN_ROOT}/programs/benchmark_dispatch.py <bench> \
--resume --dataset <path-to-dataset> --run <run-dir>
# 5. Score only after program_first_ai_review_acceptance.json says COMPLETE
python3 ${CLAUDE_PLUGIN_ROOT}/programs/benchmark_dispatch.py <bench> \
--score --dataset <path-to-dataset> --run <run-dir>
What the AI must do BEFORE invoking this command
MANDATORY first step (per the open-benchmark-keyword hook in this plugin):
- Invoke
Skill(skill="vibe-ic:open-benchmark-methodology")to load:- § 2 the run-shape decision matrix (so you don't repeat the 2026-05-28 RTLLM mistake)
- § 3 the tool-substitution disclosure obligations
- § 4 the triage rubric (A-H) — never label a fail "benchmark floor" without it
- § 5 the per-benchmark cheat sheet (current shape + status + any TARGET RE-RUN)
- Then call
benchmark_dispatch.py <bench>to see the env check + recommended commands. - Then follow the right per-shape blind instructions from
${CLAUDE_PLUGIN_ROOT}/benchmark/blind_instructions_shape_<shape>.md.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 6d ago Changed · +5 lines · -32 tokens per session 44da1cdaeef0
- 10d ago First seen · 79 lines · 118 tokens per session scan A 54b4edbe5d3f
vibe-ic-benchmark is a command published in the GitHub repository vibeic/vibe-ic (23 stars, last pushed yesterday), licensed Apache-2.0. It adds 86 tokens to every session and 1,348 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other commands, from other repositories
pcb-new
Start a new PCB project from a text description. Sources parts, generates a wired .kicadpcb, and hands off to EasyEDA for routing + JLCPCB ordering.
pcb-from-bom
Start a PCB project from an existing LCSC BOM file (CSV with C-numbers) and a design-intent description.
gf-cocotb
Generate Python testbench using Cocotb.
checklist
Generate a custom checklist for the current feature based on user requirements.
clarify
Identify underspecified areas in the current feature spec by asking up to 5 highly targeted clarification questions and encoding answers back into the spec.
specify
Create or update the feature specification from a natural language feature description.