bench

A manual command for the cc-proxy add-on that refreshes model grades and prices or measures routing speed. A proxy routes requests between an application and different model providers.

In plain words
What is it for?
Use it to run grade refreshes, speed measurements, or a speed report for the configured cc-proxy installation.
Why use it?
It provides a repeatable way to update comparison data or check how quickly requests travel through the proxy.

Command

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add commands/betmoar/cc-proxy-plugin/bench
Clone the repo
git clone --depth 1 https://github.com/betmoar/cc-proxy-plugin
Per session 37 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 1,162 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00037 $0.01162
Opus 5 $0.00018 $0.00581
Sonnet 5 $0.00007 $0.00232
Haiku 4.5 $0.00004 $0.00116

Measured yesterday against content hash edc27810aeb8, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

bench scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

commands/bench.md · 90 lines

What it actually says

Run the cc-proxy bench and show the user its output.

$1 selects the sub-command: grades (default), speed, or speed --report.

Execute:

# Resolve the plugin root. $CLAUDE_PLUGIN_ROOT is only injected in hook context,
# not into a slash-command's Bash, so fall back through the legacy PROXY_PATH
# pin, then the marketplace cache (newest version wins), then the dev repo.
# Same resolution as /cc-proxy:models — see that file for the two portability
# traps this form avoids (template substitution, and zsh word-splitting).
root=""
while IFS= read -r c; do
  [ -n "$c" ] || continue
  if [ -f "$c/scripts/bench-grades.js" ]; then root="$c"; break; fi
done <<EOF
${CLAUDE_PLUGIN_ROOT:-}
$([ -n "${PROXY_PATH:-}" ] && dirname "$(dirname "$PROXY_PATH")")
$(ls -d "$HOME"/.claude/plugins/cache/*/cc-proxy/*/ 2>/dev/null | sort -V -r | sed 's:/*$::')
$PWD
EOF
[ -n "$root" ] || { echo 'cc-proxy: cannot locate plugin root; run /cc-proxy:setup or /resume'; exit 1; }

# Trap 3 (this command's own): a slash-command body has NO real positional
# parameters. Claude Code substitutes the argument tokens TEXTUALLY before any
# shell runs, and it substitutes only $ARGUMENTS and $1 — $2 and $3 are left as
# the literal characters "$2"/"$3" for the shell to expand against an EMPTY
# argv, i.e. to nothing.
#
# Two bugs came from not knowing that, both verified against the real harness:
#   1. `shift; "$@"` forwarded nothing, silently dropping `--report` and turning
#      a read-only report into a full billed measurement run.
#   2. Reading `$1` then `$2`/`$3` was worse: for `speed --report` the harness
#      set $1 to `--report` (the LAST token, not the first), so `case "$1"`
#      matched no branch and fell through to the grades default — the wrong
#      sub-command entirely.
#
# So parse $ARGUMENTS — the whole argument string, the only token that carries
# every word — with `set --` word-splitting it into a real argv. The unquoted
# expansion is deliberate: these are shell-word arguments (`speed --report`),
# and the values are the user's own command line.
#
# `set -f` first, because unquoted word-splitting also GLOBS: measured in a
# directory holding two files, `bench speed *` split to three words
# (`speed aaa.txt bbb.txt`) instead of two. Splitting is what we want; pathname
# expansion is not, and the argv it builds is passed straight to a script.
set -f
set -- $ARGUMENTS
set +f
sub="${1:-grades}"
shift 2>/dev/null || true
case "$sub" in
  speed) node "$root/scripts/bench-speed.js" "$@" ;;
  grades) node "$root/scripts/bench-grades.js" ;;
  *)
    echo "cc-proxy: unknown sub-command '$sub' — expected 'grades', 'speed', or 'speed --report'" >&2
    exit 1 ;;
esac

Present the script's stdout verbatim — it is already formatted, and its footer states what each field means. Do not summarize, re-rank, or reword it.

Two things to keep straight if the user asks about the output:

  • grade and score are different axes. grade is the model's position in its OWN vendor's line-up, read from the vendor's version numbering — that part needs no benchmark. score is benchlm's cross-vendor number, and a trailing ~ marks it as benchlm's estimate rather than a measurement (new models are usually estimated, which is exactly when the number deserves less trust). Never read one field off the other, and never let price lower a grade.
  • speed measures the ROUTE, not the model. It never feeds grade. One ping is noise; speed --report gives median and p95 over the series, which is what makes route drift visible. A ⚠ spans >1 proxy build warning means the proxy binary changed mid-series and those numbers are not comparable.

grades needs network (benchlm.ai + OpenRouter) and writes ~/.claude/cc-proxy/grades.json. speed needs the proxy running and appends to ~/.claude/cc-proxy/speed.jsonl. On failure both say so and write nothing — a stale file is useful, a silently-empty one is a lie.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 90 lines · 37 tokens per session scan A edc27810aeb8

Subscribe to this mod's changes

bench is a command published in the GitHub repository betmoar/cc-proxy-plugin (2 stars, last pushed yesterday), licensed MIT. It adds 37 tokens to every session and 1,162 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.