skill-fleet-review

skill-fleet-review

Review every skill in the tree without reading every skill with a model.

Overview

A fleet review that reads 50 bodies with a model costs more than it returns, and most of what it would find is mechanical. So: measure everything, let the numbers pick the cohort, and spend judgment there.

The pointer axis is the exception. Descriptions are short, so reading all of them is cheap, and a trigger collision is invisible from inside one skill. It only shows up when siblings are read together, which is exactly what a fleet pass makes possible.

The rules being applied belong to docs:skill-forge and live in its bundled references.

When to use

A repository-wide sweep: before or after reorganizing plugins, when choosing what to refactor next, or on a periodic pass over accumulated drift.

Do not use it for one skill (docs:skill-audit) or to write one (docs:skill-forge).

Prerequisites

Run from the repository root. Node 18+ for the measurement script. Nothing else: no MCP server, no external CLI, and no other skill is required at any step.

Procedure

1. Measure the fleet

# Honor a caller-supplied PLUGIN_ROOT first — the abort message below tells the user to
# export it, and starting from CLAUDE_PLUGIN_ROOT would overwrite that escape hatch.
PLUGIN_ROOT="${PLUGIN_ROOT:-${CLAUDE_PLUGIN_ROOT:-}}"
[ -z "$PLUGIN_ROOT" ] && [ -d plugins/docs/skills ] && PLUGIN_ROOT=plugins/docs
if [ -z "$PLUGIN_ROOT" ]; then
  # Rank Codex cache candidates on the version basename, not the whole path: a plain
  # sort puts 0.9.0 above 0.10.0 and lets the marketplace directory outrank the version.
  cache_root="${CODEX_PLUGIN_CACHE:-$HOME/.codex/plugins/cache}"
  if sort -V </dev/null >/dev/null 2>&1; then vsort="sort -V"; else vsort="sort -t. -k1,1n -k2,2n -k3,3n"; fi
  PLUGIN_ROOT=$(ls -1d "$cache_root"/*/docs/* 2>/dev/null | awk -F/ '{print $NF "\t" $0}' | $vsort | tail -1 | cut -f2-)
fi
[ -d "$PLUGIN_ROOT/skills" ] || { echo "docs plugin root not found; export PLUGIN_ROOT" >&2; exit 1; }
MEASURE="$PLUGIN_ROOT/skills/skill-forge/scripts/measure-skills.mjs"
mkdir -p docs/audit
# Name the roots to sweep. With no argument the measurer defaults to `plugins/` when it
# exists, so a repository that also keeps skills under `.claude/skills/` gets a partial
# scan reported as repository-wide. List every root that holds skills here.
SWEEP_ROOTS="${SWEEP_ROOTS:?set SWEEP_ROOTS to every skill root to sweep, e.g. \"plugins .claude/skills\"}"
# zsh does not word-split an unquoted parameter, so a multi-root value would arrive as one
# path and measure nothing. Ask for POSIX splitting; the probe is a no-op under bash/sh.
set -o shwordsplit 2>/dev/null || true
node "$MEASURE" $SWEEP_ROOTS; rc=$?
# Exit 2 means part of the tree could not be read. Continuing would build a cohort and a
# report from a baseline that silently omits those skills — the false-clean sweep this
# skill's Pitfalls section forbids. Stop and report the unverified scope instead.
[ "$rc" -eq 0 ] || { echo "measure-skills exited $rc — scope incomplete, sweep stopped" >&2; exit 1; }

# Never overwrite an earlier run's artifacts: the first run's numbers are the baseline
# the next one is compared against. Testing for the file and then redirecting is not
# enough — two sweeps started in the same second, or in parallel, both pass the test and
# the second `>` wins. Reserve the CSV atomically instead: under `noclobber` the shell
# opens with O_EXCL, so exactly one racing run gets each candidate.
# Both artifacts share one base, so the base is what gets reserved — not just the CSV.
# Checking only `.csv` re-picked a base whose `.md` still existed (the failure cleanup
# below removes the CSV but never the report), and Step 6 then overwrote that report.
base="docs/audit/$(date +%Y-%m-%d)-fleet"; OUT="$base"; n=0
until [ ! -e "$OUT.md" ] && (set -o noclobber; : > "$OUT.csv") 2>/dev/null; do
  n=$((n + 1)); OUT="$base-$n"
  [ "$n" -gt 50 ] && { echo "measure: cannot reserve an audit path under $base" >&2; exit 1; }
done
node "$MEASURE" --csv $SWEEP_ROOTS > "$OUT.csv"; rc=$?
# Remove the reservation on failure. Leaving the truncated file behind hands the next
# reader a zero-row CSV that looks exactly like a fleet with nothing to report.
[ "$rc" -eq 0 ] || { rm -f "$OUT.csv"; echo "measure-skills --csv exited $rc — reservation $OUT.csv removed, no baseline written" >&2; exit 1; }
echo "report base: $OUT"