reproducing-ci-locally

Reproducing CI Locally

A local check is only useful if it runs the same thing the runner runs. Most "green locally, red in CI" failures are not bugs in the code — they are a difference between two commands: different paths, different test markers, different env, a different linter version, or a different interpreter.

The fix is mechanical: derive the local command from the workflow file, not from the Makefile, not from habit, not from what the last repo used.

Read the workflow before you run anything

The workflow is the contract. The Makefile is a convenience that drifts from it.

# What the gate actually is, in order
sed -n '/jobs:/,$p' .github/workflows/ci.yml

# Every command CI runs, across all workflows
grep -rn "run:" .github/workflows/

In this repo that read yields setup steps (a checkout, a toolchain, a pip install) and sh tools/verify.sh --only <slices> calls, and the gate itself is those calls: every gate job is setup plus one such call. The exception is the verified job, which runs tools/ci-verified.py (the guard that asks whether this SHA was already verified) and calls no slice. tools/verify.sh says what a slice runs. sh tools/verify.sh --list prints each slice, its tier and the steps it executes, which is the gate in this repo; gate-parity.py holds the workflow's slice names to that list.

Copy out four things, verbatim:

1. The commands and their order.
2. The paths each command is scoped to (ruff check app tests scripts is not ruff check .).
3. Test selection — marker expressions, -k filters, which suites are excluded.
4. The env: block, and the runtime/toolchain versions in setup-* steps.

Each of those four is a distinct way to get a wrong answer locally.

Paths. If CI lints app tests scripts and you run ruff check ., you get findings from directories CI never looks at — a red that isn't a merge blocker and shouldn't be "fixed" in an unrelated PR. Run it the narrow way to reproduce the gate; run it the wide way only when you're deliberately auditing.