Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add commands/iliaal/ccc-plugin/ccc-benchgit clone --depth 1 https://github.com/iliaal/ccc-pluginWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00022 | $0.00922 |
| Opus 5 | $0.00011 | $0.00461 |
| Sonnet 5 | $0.00004 | $0.00184 |
| Haiku 4.5 | $0.00002 | $0.00092 |
Grade A, and why
ccc-bench scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 71 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Run ccc retrieval regression check
Wraps ${CLAUDE_PLUGIN_ROOT}/bin/ccc-bench, which runs the benchmark runner against every *-eval.yaml corpus in the corpus directory, saves timestamped scorecards to history/, and prints a compact aggregates table.
Corpus directory resolution (first match wins):
--corpus-dir DIRargument$CCC_BENCH_CORPUS_DIRenvironment variable$HOME/ai/retrieval-bench(fallback default)
Step 1: Run the benchmark
${CLAUDE_PLUGIN_ROOT}/bin/ccc-bench $ARGUMENTS
Indexing is expected to be warm (no re-index happens here — this only runs ccc search). Each corpus takes 10-60 seconds depending on case count. For a typical 2-corpus setup, total runtime is 1-3 minutes.
If the command runs for more than ~2 minutes, run it with run_in_background: true and poll the output file.
Step 2: Report the summary
Parse the summary table the script prints and surface the key aggregates for each corpus:
- Cases count
- Miss rate (% of queries where no ground-truth file landed in top-10)
- Median first-hit rank (1 is ideal — target file came back as the top result)
- Mean recall@5 and recall@10
Flag anything that looks off — the target thresholds are miss rate ≤ 10%, median first-hit ≤ 3, recall@10 ≥ 0.70. Below those, the corpus either has stale ground truth, or something in the retrieval stack actually regressed.
Step 3: Handle corpus drift gracefully
If the script reports CORPUS DRIFT for any corpus, it means expected_files in the YAML reference files that no longer exist in the project (renamed, deleted, or moved upstream). This is not a retrieval regression — it's the benchmark corpus going stale.
Read the scorecard file (the script prints the path) to see which ground-truth files are missing. Present the list to the user and offer to:
- Update the eval YAML to remove or replace the stale references
- Skip that corpus with
--no-drift-checkfor this run only (fast mode, distorted numbers) - Leave it alone — the drift check is doing its job by flagging it
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 71 lines · 22 tokens per session scan A 0ea259179900
ccc-bench is a command published in the GitHub repository iliaal/ccc-plugin (1 stars, last pushed 4mo ago), licensed MIT. It adds 22 tokens to every session and 922 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other commands, from other repositories
cb-usage
코드브레인 활동 — Claude+Codex 토큰 + hook/MCP breakdown + PreToolUse 차단 횟수.
cb-health
코드브레인 상태 한 줄 요약 — doctor·큐·worker·인덱스.
cb-proof
코드브레인 검색 효과 검증 — legacy/v2 A/B·PPR·결정론·무증가·지연.
cb-exec
코드브레인 sandbox 셸 실행 — grep/find 등 긴 출력을 토큰 절약하며 실행.
cb-search
코드브레인 BM25 검색 — stale 자동 refresh.
cb-doctor
코드브레인 doctor — 실패한 체크만 한 줄씩.