check-tests

check-tests: Test Suite Runner & Coverage Tracker

Runs the configured test command, tracks pass/fail counts and coverage percentage over time, and alerts when regressions appear or coverage drops. READ-ONLY in terms of PRs — writes only to agent/state/test_coverage.json.

---

Safety Rules

1. HALT File — Check config.safety.halt_file first. If it exists, print reason and stop.
2. Read-only — Writes only to agent/state/test_coverage.json. Never touches test files or source.
3. No test_command — If config.test_command is empty or unset, skip gracefully.

---

Step 0: Safety

0-A: HALT Check

if file exists at config.safety.halt_file:
  Print "[HALT] check-tests stopped. Reason: {file_content}"
  EXIT immediately.

0-B: Config Validation

if config.test_command is missing or empty:
  Print "No test command configured. Skipping check-tests."
  Print "Set config.test_command (e.g. \"npm test\", \"pytest\", \"go test ./...\") to enable."
  EXIT cleanly (not an error).

---

Step 1: Load Previous State

Read agent/state/test_coverage.json. If missing, initialize with: { "schema_version": "1.0.0", "last_run": null, "run_count": 0, "status": "unknown", "results": { "total": 0, "passed": 0, "failed": 0, "skipped": 0, "coverage_pct": null }, "previous_results": null, "alerts": [], "history": [] }

Note previous passed, failed, and coverage_pct values for Step 3.

---

Step 2: Run Tests

Execute with a configurable timeout (read config.test_timeout_seconds; default 300):

{config.test_command} 2>&1

Capture exit code, stdout, stderr. Parse:

• Counts: try framework-specific patterns in order:
  • Jest: Parse each token independently from the Tests: summary line — (\d+) failed, (\d+) passed, (\d+) skipped, (\d+) total as separate optional matches (default 0 when absent). IMPORTANT: the Tests: line omits "failed" when all tests pass (e.g., Tests: 26 passed, 26 total), so a combined pattern requiring all tokens will fail.
  • pytest: (\d+) passed, (\d+) failed, (\d+) skipped, (\d+) error
  • Go: count lines matching ^ok\s+\t as passed packages, lines matching ^FAIL\t as failed packages (IMPORTANT: Go emits multiple lines containing FAIL per failed package — --- FAIL: TestName, standalone FAIL, FAIL\tpkg/path, and a trailing FAIL summary. ONLY ^FAIL\t followed by a package path represents a failed package. Similarly, --- PASS: lines are per-test, not per-package.) In verbose mode (-v), also count --- PASS: lines for individual test counts and --- FAIL: for individual test failures, reporting both: Tests: {test_passed}/{test_total} passed (packages: {pkg_passed}/{pkg_total})
  • Mocha: (\d+) passing, (\d+) failing, (\d+) pending
  • RSpec: (\d+) examples?,\s*(\d+) failures?(?:,\s*(\d+) pending)?
  • Rust/Cargo: test result: (?:ok|FAILED)\.\s*(\d+) passed;\s*(\d+) failed;\s*(\d+) ignored — Cargo emits a single summary line. ignored maps to skipped. Coverage is not emitted by default; requires cargo-tarpaulin or cargo llvm-cov.
  • Python unittest (stdlib python -m unittest): Ran\s+(\d+)\s+tests? gives total. If a line matches ^OK\b → failed=0, passed=total (OK \(skipped=(\d+)\) sets skipped). If a line matches ^FAILED\s*\((.+)\) → inside the parens read failures=(\d+) and errors=(\d+) (sum → failed) and skipped=(\d+) if present; then passed = total - failed - skipped. unittest emits NO coverage — use python3 -m coverage run -m unittest && coverage report for the pytest-cov TOTAL ... % line. (Listed — and therefore tried — BEFORE Bun, since Bun also matches Ran (\d+) tests.)
  • Bun: (\d+)\s+pass(?:\b) for passed, (\d+)\s+fail(?:\b) for failed — Bun uses present tense (pass/fail) NOT past tense (passed/failed). Also Ran\s+(\d+)\s+tests for total. (\d+)\s+skip for skipped. Coverage requires --coverage flag.
  • Vitest: Uses the same Istanbul/v8 table format as Jest for coverage. Test counts use Tests\s+(\d+)\s+passed\s+\((\d+)\) format — note: no colon after Tests, no comma separators. Parse each token independently as with Jest.
  • Fallback: generic (\d+)\s+(?:tests?\s+)?passed, (\d+)\s+(?:tests?\s+)?failed
  • Go skipped: count --- SKIP: lines for skipped tests (only visible in verbose -v mode; if not verbose, skipped count defaults to 0)
  • Compute total = passed + failed + skipped when the framework does not emit a total
• Coverage: try patterns in order:
  1. All files\s*\|\s*([\d.]+) (Istanbul/nyc table format — NOTE: data rows use bare numbers, no % sign. The first column after All files | is statement coverage.)
  2. TOTAL\s+.*?([\d.]+)% (pytest-cov)
  3. coverage:\s*([\d.]+)% (Go)
  4. Statements\s*:\s*([\d.]+)% (Jest text-summary reporter, NOT the default table)
  5. ([\d.]+)%\s*coverage (generic fallback) Use first match; if none match record null. Go multi-package note: Go emits one coverage: line per package. When multiple matches exist, compute the average across all matched values (this approximates aggregate coverage since Go does not produce a single aggregate figure). Ignore coverage: 0.0% from packages with [no test files].
• Status: exit 0 → "passing", exit 127 (command not found) or 126 (permission denied) → "error" with detail "test command not found or not executable", timeout → "error" with detail "timeout after Ns", other non-zero → "failing"