plant-defects

Plant Defects

The discipline of deliberately breaking code to prove a test can see it.

A test asserts a property of the code. Planting a defect asserts a property of the test, and the second does not follow from the first. Where a check is an external instrument, or where its claim is an absence, the second property has a bad default answer and reading the source cannot improve it.

When to Use

Use this when a check's own correctness is in question rather than the code's. The three signals, in descending order of urgency:

• The claim is an absence. No leak, no race, no unused import, no warning, nothing left behind. A search for absence succeeds for the wrong reason when the instrument was never running.
• The check is an external instrument. A sanitizer, a linter, a formatter, a leak checker, a conformance validator. Its exit code answers a question you did not ask, and its silence has two readings.
• The failure mode is a weakening rather than a break. A release store simplified to relaxed, a tolerance widened, two correct lines put in the wrong order. Each compiles and each passes, which is what separates this from a defect the compiler already refuses.

Do not use this to re-derive coverage for ordinary code whose failure is a legible compile error or an obviously failing test. The compiler is already telling you.

Core Principles

1. The second property has a bad default answer. "This test would catch that" is a claim about the test. Until it has been run against the defect it names, it is a prediction.
2. Order the assertions. Establish that the instrument was running, then that the subject ran and its output parses, and only then read the absence. What is invariant is that no absence is believed before the first two; which of them comes first depends on the instrument, and the strong form of the first (an arm built to fail) is not always available.
3. Record what happened, not what was expected. A result column filled in from the issue that filed the work is not a measurement. Several of the findings behind this skill are cases where the two differed.
4. Encoding a plant is not the same as covering it. A plant far enough outside a bound is caught by a sibling check, so the arm that names it is asserting only that the error had a detectable sign. Weaken each assertion in turn and confirm something still fails.
5. A plant that does not compile is not a passing plant. Judge on the build's exit code, not the shape of its output, or a grep for failures reads a compile error as a pass.
6. Instruments do not compose by assumption. Tabulate defect against instrument and mark what each one cannot see. Where a row turns out to be covered by nothing, say so deliberately rather than leaving it implicit. A matrix that covers every row is a fine outcome; the point is to measure the cells, not to manufacture a gap.