sumo-qa-measuring-coverage

Measuring coverage & mutation

Turn a repo's coverage/mutation run into evidence the QA report can cite. The report reads .sumo-qa/coverage.json and .sumo-qa/mutation.json; this skill produces them. The host (you) runs the tool and reads its output; the MCP tools only validate + persist.

Announce at start: "Measuring coverage/mutation and recording it for the report."

Output discipline (mandatory)

Inherits the global discipline from using-sumo-qa: never surface internal taxonomy labels; spend output on findings not framing; one question per turn; no preamble or closing pleasantries.

The Iron Law

MEASURED, REPORTED, NEVER GATED. Coverage and mutation are evidence the report shows — they never become a pass/fail gate and never flip a readiness verdict. A 100% coverage signal does not make a risk "covered" or a blocked verdict "ready". If you catch yourself treating a percentage as a safety threshold, stop.

When to Use

• "run coverage and put it in the report", "measure coverage", "what's my coverage" (when the report should carry it)
• "run mutation testing", "record the survivors", "get mutation into the scorecard"
• As a follow-on to a QA rollout, before sumo_qa_generate_qa_report, when you want the Coverage/Mutation rows populated instead of "not available".

NOT for: live PostToolUse mutation-survivor routing (that is #127's router — a different concern; see sumo-qa-strengthening-tests). Reading a saved report ≠ running the router.

Checklist

1. Detect (configured tooling only). Inspect for tooling the project already pins — coverage: pyproject [tool.coverage] / --cov in pytest addopts / .coveragerc, package.json jest --coverage / nyc / c8, go test -cover, a committed coverage.xml (Cobertura) / coverage.json (coverage.py) / lcov.info; mutation: [tool.mutmut] / mutmut, Stryker (stryker.conf.*, mutation.json), PIT (mutations.xml). Never pip install / npm i a tool. Nothing configured → stop, report "not available" (step 5). Configured but unrun → offer a scoped run; if declined, fall back.
2. Run the detected command, scoped. Run the project's own command, preferring a machine-readable report and scoping to changed code when a diff is in play (e.g. pytest --cov=<pkg> --cov-report=json; mutmut run + its results summary). A run that errors (missing dep, misconfig) → report it plainly and fall back; never mask it.
3. Collect (LLM reads any format). Read the output — JSON, XML, lcov, or console text — and extract: coverage → line_percent + a compact detail naming uncovered/weak changed files/functions (tie gaps to .sumo-qa/diff-impact.json when present — changed code, not global vanity totals); mutation → survivors + killed + a detail on where notable survivors live. Set freshness="fresh" (you just ran it), source_tool = what you ran, generated_at = an ISO-8601 timestamp.
4. Persist. Coverage → sumo_qa_record_coverage(root, coverage={...}); mutation → sumo_qa_record_mutation(root, mutation={...}). Each validates and writes the artifact, or returns an error envelope (fix the payload, don't hand-write the file). Then tell the user to (re)run sumo_qa_generate_qa_report to see the Coverage/Mutation rows populate.
5. Fallback (no tool / no run). Report concisely: "Coverage/mutation not available — no coverage tool is configured in this repo." Record nothing. A first-class outcome, never a workflow failure, never a reason to invent a number.