mutation-testing

Mutation Testing

Mutation testing verifies whether your tests detect bugs, not just whether they execute code. A "mutant" is a small deliberate change (flip > to >=, delete a return) applied to the source. If your tests pass with the mutant, they would not have caught that bug.

Deeper reference: patterns/mutation-testing/MUTATION_TESTING.md (full operator list, equivalent mutants, cost management strategies).

---

When to use it

• After reaching line/branch coverage targets but before shipping critical business logic (payment, auth, data validation).
• When a production bug slips through a test suite that claims high coverage.
• As a periodic quality audit (weekly/monthly) rather than a per-commit gate.

When to skip: Prototype and Internal tiers; generated code; pure data classes with no conditional logic; large legacy codebases where the cost of triaging hundreds of mutants outweighs the benefit.

---

Mutation score

$$\text{mutation score} = \frac{\text{killed}}{\text{total} - \text{equivalent}} \times 100%$$

Score	Verdict
≥ 80%	Strong; surviving mutants warrant individual inspection
60–79%	Adequate; identify surviving categories and add targeted tests
Below 60%	Test suite quality is insufficient; line/branch percentages alone are misleading

---

Python — mutmut

pip install mutmut
mutmut run --paths-to-mutate src/billing.py
mutmut results          # list surviving mutants
mutmut show 15          # show the diff for mutant #15

---

TypeScript / JavaScript — Stryker

npm install --save-dev @stryker-mutator/core @stryker-mutator/jest-runner
npx stryker init
npx stryker run

stryker.config.js:

module.exports = {
  testRunner: 'jest',
  coverageAnalysis: 'perTest',  // only test mutants actually covered
  mutate: ['src/billing/**/*.ts'],
  thresholds: { high: 80, low: 60, break: 50 },  // break = CI gate
};

---

Go — gremlins

go install github.com/go-gremlins/gremlins/cmd/gremlins@latest
gremlins unleash ./internal/billing/...