ai-agents ci-scripts.instructions.md

CI and Validation Script Rules

Scripts under scripts/validation/, build/, and .github/workflows/ gate every PR. A broken change here blocks the entire repository (see Issue #1711).

MUST

1. Local run before commit. CI-critical scripts MUST be exercised locally before commit. Use gh act for workflows, direct uv run python invocation for validation scripts, and the actual test suite for helpers.
2. Shift-left validation. Before pushing, MUST run uv run python scripts/validation/pre_pr.py and resolve any failures.
3. Python for new scripts. New scripts MUST be Python per ADR-042. MUST NOT create new *.sh bash scripts.
4. Exit codes. Scripts MUST follow the exit code contract: 0=ok, 1=logic, 2=config, 3=external, 4=auth (AGENTS.md).
5. Tests required. New validation scripts MUST have tests under tests/. Do not add CI tests to shipped skill directories.
6. Pin Actions to SHA. Workflow changes MUST pin every Action reference to a commit SHA.
7. Verify worktree identity before writing. A script that resolves the repository root and then writes to it MUST confirm the current directory is inside the resolved root before the first write (Path.cwd().resolve().is_relative_to(top_level)). git rev-parse --show-toplevel reports a claim, not a fact about where you are: a local core.worktree value or a GIT_WORK_TREE environment variable redirects it to a directory you are not standing in, and git status then reports every tracked file as deleted because it is looking somewhere else. Measured: an ordinary git worktree add sets neither, a moved worktree still resolves correctly, and a worktree whose main checkout moved away fails closed with a non-zero exit. So the redirection is always something a person or a tool set on purpose, which is exactly why a script that inherits it has no way to notice.
8. Anchor helper resolution on the absolute top level. A resolver that walks candidate roots to find a repository helper MUST anchor its in-repo rung on git rev-parse --show-toplevel, and MUST order that rung ahead of any out-of-repo root. A bare relative .claude rung only resolves when cwd happens to be the repository root; invoked from a subdirectory it falls through to a copy under ~/.copilot/installed-plugins or ~/.claude/plugins/cache, which can be arbitrarily old. check_skill_resolver_anchoring.py enforces this for SKILL.md resolvers; the same requirement binds resolvers written anywhere else, where nothing enforces it for you.
9. Read the state you are asserting about, and name the ref. A claim about what the repository contains MUST be computed from a named ref: git ls-tree -r -z --name-only HEAD for a path inventory, the full git ls-tree -r -z HEAD wherever entry mode matters, and git log HEAD for history. Use -z; paths are not newline-safe, and --name-only hides modes, so the tracked memory_enhancement symlink is indistinguishable from a regular file. Such a claim MUST NOT come from git log --all or from a directory walk. --all reads every ref the clone holds rather than the branch: at diagnosis this clone held 2054 refs/remotes/pr/* refs while remote.origin.fetch covered only branch heads, and deleting one of them flipped a shipped test from failing to passing without changing a byte of the repository (Issue #3753). Prefer HEAD to origin/main, since a guard scoped to the base branch cannot see what the current change does. Reads of the working tree, the index, and untracked files remain correct and required wherever that state is itself the subject, as in regeneration drift and pre-commit checks. A ratchet baseline is a claim about a ref, so the measurement behind it MUST NOT read untracked state: Path.exists() counts gitignored build output that the author happens to have generated and CI never will, so the same commit scores differently on two machines. Measured 2026-08-07: a baseline recorded with build/audit/GENERATION-AUDIT.md present landed one too low on four entries and turned main red for every open PR, while the identical command on the identical commit passed locally (Issue #4748). Their findings describe local state and MUST NOT be restated as claims about a ref: a directory walk reported three skills as unusable when what remained on disk was untracked residue from a deletion in PR #2359, and the resulting Issue #3420 was closed NOT_PLANNED.
10. Convert every failure signal into a non-zero exit before the step ends. When a run: block moves into a Python module under ADR-006, the shell semantics it is replacing MUST be preserved at the boundary: under set -e any non-zero command aborted the step, so the module MUST return a non-zero code to sys.exit for the same conditions. Returning a findings list, an error string, None, or False to a caller that ignores it converts a red step into a green one, and the extraction is then a silent-pass detector rather than a check. Six confirmed instances are tracked in Issue #4068. A green step whose behavior changed in this direction is worse than the shell it replaced, because the shell failed loudly and the module reports success. Verify by running the module against input known to be bad and reading $?, not by reading the log.
11. Convert every detected violation into a non-zero exit. A script that detects a violation and prints a message but exits 0 has the same observable behavior as a script that found nothing. Hooks and CI steps read the exit code; they do not parse output. If the script found a problem, it MUST exit non-zero. If it found nothing to check (empty input set, no files matched), it MUST exit 0 and SHOULD print a count of examined items so a caller can tell the difference between "zero violations in N items" and "zero violations because nothing was examined".
12. Distinguish a run that did nothing from a run that succeeded. A workflow, checker, or gate that early-returns when there is no work MUST NOT report that outcome the same way it reports completed work, or the signal inverts: the job goes green exactly when it is idle and red exactly when it acts, and the failure hides inside a mostly-green history. Always print the examined count alongside the violation count: "0 violations in 381 files" is verifiable; "OK" is not. A mutation harness MUST report DID-NOT-APPLY when the target literal is absent so that a moved or renamed target does not become an undetected surviving mutant.
13. A PR introducing a gate MUST demonstrate the gate passing against the full corpus before merge. A unit test over fixtures proves the checker's logic; it proves nothing about whether the existing corpus satisfies the gate. Those are separate claims and only the second determines whether main goes red. The PR body or a PR comment MUST quote the output of the gate's own command run against the full corpus on the PR branch. A gate that ships with known outstanding violations blocks every subsequent push by every contributor and must not merge. Measured cost: two violations in a single episode file blocked the entire repository for a multi-hour window after PR #4219 merged, driving three hook-bypass attempts, each of which is a policy violation under ADR-086:95-98 (Issue #4262).