Characterization Test Writer

Characterization Test Writer

Pin what legacy code currently does - bugs included - so a refactor that changes behavior fails loudly instead of silently. The costly mistake this prevents is the "clean" refactor that quietly changes an output nobody noticed was load-bearing, discovered weeks later in production. A characterization test records reality; it never judges it.

Operating procedure

Order matters: the seam must exist before you can capture output, output must be captured before you can stabilize it, and coverage is only measurable once tests run.

Step 1: Gather inputs

• The exact function/module about to change, and the refactor's blast radius - which code paths will the change touch? Only those need pinning.
• The dependency list: database, clock, RNG, network, filesystem, globals, environment. Each is a nondeterminism source to control in Step 4.
• Any recorded production inputs or logs available for replay. Label assumptions about "typical inputs" as guesses.

Step 2: Find the smallest seam

Untested legacy code reaches for databases, clocks, network, and globals, so it usually cannot be called in isolation. Introduce the minimum seam that lets you invoke it: extract a method, parameterize a constructor, inject the dependency, or subclass-and-override (the "subclass to sense" technique from Working Effectively with Legacy Code). Do not refactor the body before the net exists - seam-introduction edits must be mechanical and behavior-preserving (rename, extract, inject), nothing that changes logic.

Step 3: Capture actual output, not expected

Write a deliberately failing assertion, run it, read the real value from the failure message, and paste it in:

def test_characterize_shipping_for_oversize_order():
    result = calc_shipping(weight_kg=32, region="EU")
    assert result == "PLACEHOLDER"   # run -> failure says: AssertionError: 47.6 != PLACEHOLDER

becomes, after one run:

def test_characterizes_oversize_eu_shipping():
    result = calc_shipping(weight_kg=32, region="EU")
    assert result == 47.6  # observed 47.6, not the 45.0 the spec claims - see pins_known_bug below