test-baseliner

Run the project's full test suite and return a structured result for regression comparison.

Operates in two modes. The caller must specify which mode to use.

Mode: capture

Run the test suite and return a baseline snapshot. Use this before making changes.

Steps

1. Detect framework — Search the working directory for build/config files in this order:

   • pom.xml → Maven, command: mvn test -q
   • build.gradle or build.gradle.kts → Gradle, command: ./gradlew test (fall back to gradle test if no wrapper)
   • package.json → read the scripts.test field; if absent, use npm test. Check for a workspaces field — if present, use npm test --workspaces --if-present.
   • pyproject.toml, setup.py, or pytest.ini → pytest, command: pytest -v
   • Makefile containing a test target → make test
   • If no framework found: return the structure below with Framework = "not detected", all counts = 0, and a note explaining no runner was found. Do not fail.

2. Run — Execute the detected command. Allow up to 10 minutes. Capture stdout and stderr combined. If the run aborts (non-zero exit, truncated output, or unrecognized runner output) with no parseable pass/fail counts, treat this as a failed run for the Status computation in step 4 (do not fail the tool call — still return the structure).

3. Parse — Extract from the output:

   Framework	Passing count	Failing count	Skipped count
   Maven	Tests run: X minus failures+errors per module, summed	Failures: Y, Errors: Z summed	Skipped: N summed
   Gradle	X tests completed minus failed	, Y failed	, Z skipped
   pytest	X passed	Y failed or Y error	N skipped
   Jest/npm	X passed	Y failed	Y skipped
   Make	best-effort: look for any X passed / X failed / X pass / X fail patterns. If no pattern is found, set counts to 0 and include a note.	same	same