benchmark-verify

Benchmark Verify — normalized doc→production-ready verification

We run benchmark ICs to validate the whole Vibe-IC flow: that starting from a complete set of Design Documents we can reach a production-ready result. A benchmark run is not finished when the flow returns a verdict — it is finished when this skill's five pillars all pass and the single report is written.

This skill is chip-AGNOSTIC and MANDATORY for every benchmark IC. Apply it to each IC under benchmark_clean/<ic>/ (and any future benchmark set).

⛔ Prerequisite — honesty + the corrected protocol

Read benchmark_clean/METHODOLOGY.md first. Non-negotiable:

• Input = Design Documents ONLY (incl. explicit PDK); nothing produced by a later phase is fed back as input.
• Phase-2 RTL must be GENERATED; IP reuse is allowed but every reused source MUST be tagged REUSED-IP in SOURCE_MANIFEST.md (vs GENERATED). Production-readiness credit applies only to GENERATED content; reused content is reported separately.
• No vacuous result counts as PASS (e.g. a Magic gds read that dropped geometry and reports "0 DRC" is INCONCLUSIVE, not clean). A missing input is PENDING, never a silent PASS. Enforced by programs/drc_vacuous_pass_check.py — flips a 0-violation DRC verdict to INCONCLUSIVE unless the log proves geometry was loaded; SKIPs (never PASSes) when no DRC log exists.
• The upstream open-source IP is the golden oracle for cross-check only — never a Phase-1/2 input.

The five pillars (== report sections == hard gates)

Pillar 1 — Functional Verification Coverage ▸ gate: == 100%

LLM judgment (irreducible): walk L1–L27 (functional spec, interface, register map, command/protocol, timing, behavioral sequences, test cases) and enumerate every requirement — a program cannot reliably know that a prose timing sentence in L8 is a distinct testable requirement (vs a restatement) nor author the directed test that covers it. For each requirement, bind a verification item (directed test, assertion, golden-vector check, or formal property) and confirm it PASSES against the generated RTL.

• Functional Coverage = verified_requirements / total_requirements → must be 100% (the == 100% gate is enforced by programs/benchmark_verify_report.py).
• Emit reports/functional_coverage.json: {"requirements":[{"id","source","desc","status":"PASS|FAIL|PENDING"}]}.
• Closed-loop: if < 100%, write the missing tests and/or fix the RTL (use rtl-repair) and re-verify until 100%. Do not waive a requirement; if a doc requirement is genuinely untestable, that is a spec defect to record, not a pass.