running-quality-checks

Running Quality Checks

The verifier's procedure: detect the checks the project configures, run them in speed order, and report evidence. No opinions — just evidence.

Process

1. Detect available checks. Inspect project configuration to find runnable checks:

   • package.json scripts (format, lint, typecheck, build, test)
   • Makefile targets
   • CI configuration (.github/workflows/, .circleci/, etc.)
   • Tool configuration files (.eslintrc, tsconfig.json, prettier.config, biome.json, etc.)

2. Run checks in speed order. Execute each detected check, fastest first:

   1. Format — Prettier, Biome format, or equivalent (--check mode)
   2. Lint — ESLint, Biome lint, Clippy, or equivalent
   3. Type check — TypeScript tsc --noEmit, mypy, or equivalent
   4. Build — Production build command
   5. Test — Test suite execution

   Checks interfere. A production build and a dev-server-backed browser suite share one build directory in most frameworks that have one, so back-to-back in a single sequence the second reads state the first wrote and fails on assertions that read exactly like regressions. Clear the build directory and run the browser suite alone. A suite that boots its own servers is unsafe beside anything else, another agent's dev server included.

3. Capture results. For each check, record:

   • The exact command run
   • The exit code
   • For failures: the relevant error output (trimmed to essential lines)
   • For passes: one-line confirmation

Verdict Logic

• PASS — All detected checks passed (at least one check must exist).
• FAIL — One or more detected checks failed. List every failure.
• FAIL — No checks detected at all. A project with zero configured quality checks (no linter, no type checker, no test suite, no build) cannot pass verification. Report what is missing and recommend configuring at least format, lint, and test scripts.

Rules

• Run every check you can detect. A check the project does not configure is not a detected check.
• Do NOT fix failures. Report them exactly as they occur.
• Do NOT interpret results beyond pass/fail. No suggestions, no opinions.
• Keep output concise. For failures, include only the lines needed to understand what went wrong. Do not dump entire build logs.
• If a check hangs for more than 120 seconds, kill it and report TIMEOUT.
• Do NOT retry to mask intermittent failures. Each check runs once. If a test or check fails, report it. If you happen to know the same test passed in a previous run (e.g., the orchestrator re-dispatched after a code fix), note the intermittency in the report (### Notes — Intermittent: testFoo passed on retry, the underlying race condition is unresolved). Reruns that turn red → green without a code change are evidence of a flake or a real intermittent bug, not a verdict of PASS.
• A baseline is comparable only under the same isolation. When this run is the before side of a before/after comparison (principle-pre-image-first), run both sides the same way. A false red recorded as the pre-change state reclassifies a later regression as pre-existing — a failure in the safe direction, which is why it goes unnoticed.
• Coverage is reported, not gated. If the project has a coverage tool configured, run it and report the coverage delta for changed files (e.g., "coverage on changed files: 73% → 78%"). Do NOT gate on an absolute coverage threshold. Coverage tells you what is NOT tested. It does not tell you what IS tested is good. Pair with mutation testing when available. Require coverage to trend upward rather than mandating a fixed threshold.