Getting it into your agent
It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.
git clone --depth 1 https://github.com/vibeic/vibe-icnpx agentmods add skills/vibeic/vibe-ic/open-benchmark-methodologyWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/vibeic/vibe-ic/open-benchmark-methodology)<a href="https://agentmods.dev/skills/vibeic/vibe-ic/open-benchmark-methodology"><img src="https://agentmods.dev/badge/skills/vibeic/vibe-ic/open-benchmark-methodology/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/vibeic/vibe-ic/open-benchmark-methodology"><img src="https://agentmods.dev/badge/skills/vibeic/vibe-ic/open-benchmark-methodology.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00114 | $0.05970 |
| Opus 5 | $0.00057 | $0.02985 |
| Sonnet 5 | $0.00023 | $0.01194 |
| Haiku 4.5 | $0.00011 | $0.00597 |
Grade A, and why
open-benchmark-methodology scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 512 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Open-Benchmark Methodology
The benchmark measures the Vibe-IC product path, not a benchmark-tuned agent. Every runnable RTL evaluation uses the same solve core as an ordinary IC-design request. Dataset differences are allowed only at the input/output boundary.
Rule 0 — one product entry
For the open RTL suites in benchmark/BENCHMARK_REGISTRY.json, the only
authoring path is:
benchmark_dispatch.py <bench> --solve
-> benchmark_io_adapter.stage (input translation only)
-> task_nature_route (prompt/context evidence only)
-> vibe_ic_one_shot_runner --entry-step (normal product runner)
-> Program candidate or declared AI backup
-> independent hash-bound AI review
-> benchmark_dispatch.py <bench> --resume
-> accepted candidate
-> benchmark_dispatch.py <bench> --score
-> official host scorer
For whole-IC benchmark_clean, use the ordinary /vibe-ic-all entry,
vibe_ic_one_shot_runner.py, and the benchmark-verify skill. A benchmark IC
must not select another solver because of its IC name.
Forbidden alternatives include:
- benchmark-name or problem-id routers, task loops, tier pipelines, solve pipelines, Phase-1 wrappers, prompt exporters, or free-hand authoring loops;
- choosing a route from a CVDP
cid, benchmark label, problem name, expected answer, prior score, or hidden harness; - writing scoreable RTL directly and then calling a gate/scorer;
- publishing a partial previous-failure run as a benchmark result.
benchmark/benchmark_entry_surface_check.py enforces the shipped entry surface
and is called by --solve. If it fails, stop and repair the product entry before
running a benchmark.
GENERAL-CORE / THIN-ADAPTER
Benchmark-specific code may only translate formats or invoke the official scorer. A thin adapter may:
- stage the current record's visible prompt and supplied context into a normal project;
- collect an already-produced candidate;
- package exact accepted bytes into the scorer's required envelope;
- run image/tool preflight and translate official verdict files.
What ships with it
2 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday Changed · +51 lines f97c5eb36a9b
- 3d ago Changed · +70 lines 75e76bdaea70
- 4d ago Changed · +4 lines d0a5f70a0741
- 5d ago First seen · 387 lines · 114 tokens per session scan A ea5bd98cc1df
open-benchmark-methodology is a skill published in the GitHub repository vibeic/vibe-ic (23 stars, last pushed today), licensed Apache-2.0. It adds 114 tokens to every session and 5,970 once invoked, about $0.0006 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
analog-netlist-crawl
Crawl and analyze post-layout parasitic netlists without running SPICE. Answers "what's the effective resistance from node A to node B across this massive R mesh?", "inside the VREFN mesh, which device pins are electrically farthest apart?", "which nets have the worst coupling?", "where does settling bottleneck?" — by…
analog-design
Transistor-level circuit design for one analog sub-block. Produces Spectre netlist with hand-calculation rationale. Use when designing a specific circuit block after architecture is defined.
analog-verify
Pre-simulation review and Spectre simulation verification for analog circuits. Reviews circuit netlist and testbench, runs simulation, produces margin report. Use after analog-design completes a netlist.
analog-pipeline
MANDATORY — MUST load this skill when the user mentions: OTA, ADC, PLL, comparator, bandgap, LDO, amplifier, opamp, or any analog/mixed-signal IC design task. Full analog design pipeline: spec -> architecture -> design -> verify -> deliver. Orchestrates analog-decompose, analog-behavioral, analog-design…
analog-audit
Audit analog circuit netlists for correctness, quality, and risks. Supports both pre-layout (schematic) and post-layout (extracted) netlists. Post-layout mode filters massive parasitic netlists before auditing. Works without EDA. TRIGGER on: "audit", "review netlist", "check this circuit", "design review"…
analog-decompose
Architect Phase 1: decompose top-level analog spec into sub-block specs. Use when starting a new analog design to select architecture, allocate budgets, define sub-block specs, write testbenches, and create verification plans.