benchmark-models

When to invoke

Use when: "benchmark models", "compare models", "which model is best for X", "cross-model comparison", "model shootout".

Preamble

eval "$(~/.vibestack/bin/vibe-slug 2>/dev/null)" 2>/dev/null || SLUG="unknown"
_LEARN_FILE="${VIBESTACK_HOME:-$HOME/.vibestack}/projects/${SLUG:-unknown}/learnings.jsonl"
if [ -f "$_LEARN_FILE" ]; then
  _LEARN_COUNT=$(wc -l < "$_LEARN_FILE" 2>/dev/null | tr -d ' ')
  echo "LEARNINGS: $_LEARN_COUNT entries loaded"
  if [ "$_LEARN_COUNT" -gt 5 ] 2>/dev/null; then
    ~/.vibestack/bin/vibe-learnings-search --limit 5 2>/dev/null || true
  fi
else
  echo "LEARNINGS: none yet"
fi

{{include lib/snippets/session-host.md}}

{{include lib/snippets/decision-brief.md}}

{{include lib/snippets/working-protocols.md}}

{{include lib/snippets/state-protocols.md}}

/benchmark-models — Cross-Model Skill Benchmark

Different from /benchmark — that skill measures web page performance (Core Web Vitals, load times). This skill measures AI model performance on skills or arbitrary prompts.

---

Step 0: Locate the binary

BIN="$HOME/.vibestack/bin/vibe-model-benchmark"
[ -x "$BIN" ] || { echo "ERROR: model benchmark binary not found." >&2; exit 1; }
echo "BIN: $BIN"

If not found, stop and tell the user: "vibe-model-benchmark is required for this skill but is not installed at ~/.vibestack/bin/vibe-model-benchmark. vibestack does not bundle this binary — it's a separate dependency. See docs/external-tools.md for current options."

---

Step 1: Choose a prompt

Use AskUserQuestion with the preamble format:

• Re-ground: current project + branch.
• Simplify: "A cross-model benchmark runs the same prompt through 2-3 AI models and shows you how they compare on speed, cost, and output quality. What prompt should we use?"
• RECOMMENDATION: A because benchmarking against a real skill exposes tool-use differences, not just raw generation.
• Options:
  • A) Benchmark one of my skills (we'll pick which skill next). Completeness: 10/10.
  • B) Use an inline prompt — type it on the next turn. Completeness: 8/10.
  • C) Point at a prompt file on disk — specify path on the next turn. Completeness: 8/10.