pick-integ

Integ Test Picker

Decide which integration tests to run right now, ranked by two signals: how stale the last run is (from the committed ledger) and whether recent changes touch the area a test exercises. It RECOMMENDS only; the orchestrator runs the chosen tests via /run-integ.

The ranking is a running ORDER, not a budget (CLAUDE.md → "Cost is not a tiebreaker"): when several tests plausibly cover the touched code, run all of them, and prefer a broad-set fixture whenever the change is cross-cutting. Priorities decide what runs FIRST; a plan too long for one session is handed forward per "Running a large plan across sessions" — never silently truncated, and the wrap-up names exactly which tests were not run.

Inputs

• Optional positional base-ref (default origin/main): the diff base. Use the last release tag when picking post-merge; origin/main for a branch.

Data sources

1. Ledger docs/_generated/integ-last-run.tsv (committed, one row per test; written by /run-integ on every run):

   • Stale: age > the integ-gate TTL (14 days) — past that the gate markers expire, so a clean result no longer proves today's AWS behavior.
   • Expiring soon: age 12–14d. The ledger accumulates in sweep-shaped cohorts (one sweep stamps 100+ rows with one timestamp), so a strict >14d boolean answers "nothing" or "everything" — always report this tier; "zero stale" is misleading when 105 tests cross the TTL tomorrow (issue #1508).
   • Failing: result == FAIL — always a candidate.
   • Never-run: a fixture directory with NO ledger row — highest staleness.

2. Recent changes: git diff <base-ref>...HEAD --name-only, mapped to tests via the table below.

Steps

1. Discover the universe + ledger state:

   LEDGER="docs/_generated/integ-last-run.tsv"
   now=$(date -u +%s)
   # Union-merge can leave duplicate rows — LAST row per test wins. The
   # header is MULTIPLE `#` lines; skip them all (a single NR==1 guard let
   # later header lines through as phantom stale rows).
   DEDUPED="$(mktemp)"
   awk -F'\t' '/^#/{next} {last[$1]=$0} END{for (t in last) print last[t]}' "$LEDGER" > "$DEDUPED"
   # never-run: fixtures with no ledger row
   comm -23 \
     <(ls -d tests/integration/*/ | sed 's#tests/integration/##;s#/##' | sort) \
     <(awk -F'\t' '{print $1}' "$DEDUPED" | sort)
   # resolve each row's age ONCE, then slice for the views below.
   AGES="$(mktemp)"
   awk -F'\t' -v now="$now" '{
     cmd="date -u -j -f %Y-%m-%dT%H:%M:%SZ \""$2"\" +%s 2>/dev/null || date -u -d \""$2"\" +%s 2>/dev/null";
     cmd | getline t; close(cmd);
     printf "%s\t%d\t%s\t%s\n",$1,int((now-t)/86400),$3,$6
   }' "$DEDUPED" > "$AGES"
   
   # age histogram — makes a clustered cohort visible even when NOTHING is >14d
   awk -F'\t' '{print $2}' "$AGES" | sort -n | uniq -c |
     awk '{printf "%6d tests at age %sd%s\n",$1,$2,($2>14?"   <-- STALE":($2>=12?"   <-- crosses the TTL in "(15-$2)"d":""))}'
   
   # stale (>14d) or failing:
   awk -F'\t' '$3=="FAIL" || $2>14 {printf "%s\tage=%sd\tresult=%s\t%s\n",$1,$2,$3,$4}' "$AGES"
   
   # expiring soon (12-14d, not already failing) — the cliff cohort:
   awk -F'\t' '$2>=12 && $2<=14 && $3!="FAIL" {printf "%s\tage=%sd\texpires in %dd\n",$1,$2,15-$2}' "$AGES"
   
   # one-line cliff summary for the step 4 header:
   awk -F'\t' '$2>=12 && $2<=14 {n[15-$2]++} END{for (d in n) printf "%d\t%d tests expire in %dd\n",d,n[d],d}' "$AGES" |
     sort -n | cut -f2-
   

   (The date line handles both BSD/macOS -j -f and GNU -d.)