validate-recipe

Validation recipe — pr-review

Stack

TypeScript CLI (Node >= 20), esbuild single-file bundle at dist/cli.cjs; tests via node:test + tsx.

Tier 1 — static

• npm run build # tsc typecheck + esbuild bundle; no separate lint script exists

Tier 2 — tests

• Full suite: npm run test # scripts/test.mjs → node --test --import tsx, recursive over tests/**/*.test.ts
• One file: node --test --import tsx tests/<file>.test.ts

Tier 3 — runtime

• The surface is the bundled CLI: node dist/cli.cjs <command> (rebuild first).
• Publication-policy offline fixtures: after building, node --test --import tsx tests/exit-code.test.ts drives the bundled CLI with a deterministic attempt-writing runtime, all five severities, full verifier/dedupe/artifact flow and verify --offline. It also drives standalone post through mocked HTTP for all three input shapes and proves input bytes stay unchanged. Verified locally: 9 tests pass. These are not real-provider publication evidence; the publication acceptance case in evals/acceptance/README.md still needs authorized provider credentials.
• Offline full-pipeline dogfood (no PR, no network beyond packs): npm run build then npm run dogfood -- --base origin/main — converts the branch diff into a synthetic gather and drives the real bundle with --from-gather --dry-run. Add --context-only for routing only. Refuses a stale bundle, so build first. URL-based companions are off here by design.
• Full pipeline with no LLM — a stub runtime. --copilot <path> overrides the binary and pins the copilot runtime, so a stub does the whole real run: node dist/cli.cjs review <url> --from-gather <gather.json> --dry-run --no-codex --no-companions --copilot <stub.cmd>. On win32 the stub is a .cmd doing node "%~dp0stub.mjs" %* — %~dp0 is right ONLY when the stub is launched by path, as --copilot <stub.cmd> does; a stub found through PATH (a claude.cmd for companion detection, see the 2026-10-15 addenda) must name its script by absolute path; the stub reads --add-dir from argv to find the run dir, parses dispatch-plan.json, and writes attempt-<n>.json under each reviewer.attemptsDir (plus reviewer.capabilityPath for installed-plugin passes). Batches arrive as separate invocations — count them in a file under the run dir, since the CLI numbers attempts itself. Withhold one reviewer's attempt file to force the automatic-recovery batch. The child's stderr is captured, not passed through, so assert on run artifacts rather than on the stub's own output. This is the only way to exercise dispatch, recovery and the audit paths offline; pair it with git show origin/main:dist/cli.cjs > <tmp>/cli-main.cjs for a same-scenario before/after.
• Side-effect-free smokes: node dist/cli.cjs cache clear --pr <url> (proves URL parse end to end), a bad URL with --detach (must fail foreground, exit 2, no new dir under ~/.pr-review/runs), node dist/cli.cjs packs sync / packs list / doctor (packs clone + freshness + Linguist cache, no PR needed).
• Offline pass-selection smoke: in a temp repo with .pr-review.yaml containing skill_packs: [], run node dist/cli.cjs review <url> --context-only — must exit 2 (zero passes on a code PR) while still rendering the ## Stack section.
• Live smokes (gated on credentials — gh/az): --context-only against 1 real GitHub PR and 1 real ADO PR — must exit 0, ## Stack plausible for the diff, ## Passes non-empty with the expected glob hits.
• File-list completeness smoke (gh only, read-only, ~30 API calls): from a directory that is NOT a checkout, node dist/cli.cjs gather https://github.com/OpenAPITools/openapi-generator/pull/24767 --no-cache must exit 1 with github listed 3000 of 9782 changed files — file list truncated … run git fetch origin master refs/pull/24767/head there and retry and leave nothing under ~/.pr-review/cache/github/OpenAPITools__openapi-generator/.
• Completion-path live proof (not run here — needs a clone of a >3000-file PR): clone OpenAPITools/openapi-generator, git fetch origin master refs/pull/24767/head, then the same gather command from the clone — PASS when stderr says completed 6782 file(s) from git at <root> and the JSON holds 9782 entries. Until someone runs it, the completion path's live evidence is the temp-git-repo tests in tests/gather-cache.test.ts.
• Cache-marker smoke: after a cached gather of a real PR, delete changedFilesComplete from the entry under ~/.pr-review/cache/<provider>/<scope>/<n>/, run gather <url> again — stderr must say cache entry predates the file-list completeness check — refetching, the entry is rewritten with the marker, and a third run is a cache hit.
• pr-review-gather.json after a GitHub gather with --no-cache: metadata.changedFileCount equals changedFiles.length, changedFilesComplete is true, and there is no fullDiff key (retired in #26). Only on a fresh gather — a cache hit returns the stored payload verbatim, so an entry written by <= 0.11 still yields a fullDiff, which is expected and harmless.
• Real (dispatching) reviews need provider auth AND a runtime on PATH — treat as not locally verifiable unless both exist. Run them from a COPY of the bundle (cp dist/cli.cjs $TEMP/cli-x.cjs) so a later rebuild cannot abort the run.
• Config-dependent CLI smokes without touching the real ~/.pr-review: run the bundle with USERPROFILE=<tmp-home> HOME=<tmp-home> (Node's homedir() follows USERPROFILE on Windows) and a .pr-review/config.yaml written under it; env -u GITHUB_TOKEN -u GH_TOKEN … makes the auth outcome deterministic. Before/after against the pre-change bundle: git show origin/main:dist/cli.cjs > <tmp>/cli-main.cjs — the committed bundle is fresh by CI contract, so no worktree build is needed.