A local benchmark workflow for running the repository's vibe evaluation and maintaining its nightly baseline. A benchmark is a repeatable set of checks used to compare software results over time.

Use it to run the on-demand benchmark, operate the scheduled nightly evaluation, or compare results against the local baseline.

It provides a consistent way to measure changes without treating the benchmark as a per-pull-request or continuous-integration requirement.

vibe-bench — Skill for Claude Code and Codex by closedloop-ai

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add closedloop-ai/claude-plugins --skill vibe-bench

Clone the repo

git clone --depth 1 https://github.com/closedloop-ai/claude-plugins

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/closedloop-ai/claude-plugins/vibe-bench/github.svg)](https://agentmods.dev/skills/closedloop-ai/claude-plugins/vibe-bench)

Your own site

<a href="https://agentmods.dev/skills/closedloop-ai/claude-plugins/vibe-bench"><img src="https://agentmods.dev/badge/skills/closedloop-ai/claude-plugins/vibe-bench/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/closedloop-ai/claude-plugins/vibe-bench"><img src="https://agentmods.dev/badge/skills/closedloop-ai/claude-plugins/vibe-bench.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 22 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 127 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00022	$0.00127
Opus 5.5	$0.00009	$0.00051
Sonnet 5.5	$0.00004	$0.00025
Haiku 4.5	$0.00002	$0.00013

Measured 8d ago against content hash 7c052ff64f1c, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

vibe-bench scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.agents/skills/vibe-bench/SKILL.md · 11 lines

What it actually says

Vibe benchmark

Read and follow .claude/skills/vibe-bench/SKILL.md in full. That file is the canonical workflow. The on-demand start-to-PR benchmark is Claude Code only. The scheduled change-day nightly runs the full eval corpus on Claude Code and Codex. Neither is a per-PR or CI requirement.

Nightly Slack delivery uses the authenticated Slack MCP relay through headless Claude Code; no Slack bot-token file or scheduler environment file is required.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

8d ago Changed · +2 lines 7c052ff64f1c
9d ago First seen · 9 lines · 22 tokens per session scan A 3ed3bf8f92ac

Subscribe to this mod's changes

vibe-bench is a skill published in the GitHub repository closedloop-ai/claude-plugins (124 stars, last pushed today), licensed Apache-2.0. It adds 22 tokens to every session and 127 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-31.

Analytics and metrics Claude Code LLM Testing agentic-ai ai-agents anthropic developer-tools

vibe-bench next to 6 skills from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
vibe-benchclosedloop-ai/claude-plugins	Skill	124	today	Apache-2.0	A	22	Claude Code, Codex
dcf-modelNousResearch/hermes-agent	Skill	262k	today	MIT	A	14	Claude Code, Codex
grounded-citationsNousResearch/hermes-agent	Skill	262k	today	MIT	A	16	Claude Code, Codex
openhandsNousResearch/hermes-agent	Skill	262k	today	MIT	A	19	Claude Code, Codex
react-patternsaffaan-m/ECC	Skill	299k	2d ago	MIT	A	49	Claude Code, Kiro
mcporterNousResearch/hermes-agent	Skill	262k	today	MIT	A	16	Claude Code, Codex
apple-notesNousResearch/hermes-agent	Skill	262k	today	MIT	A	17	Claude Code, Codex