property-based-testing

Property-Based Testing — Test Once, Verify Against Thousands

Core Concept

Instead of writing specific test cases, you write properties — statements about what your code MUST do for ALL inputs:

Unit test:  "sort([3,1,2]) == [1,2,3]"         → 1 case
Property:   "sorted(xs) == sorted(sorted(xs))"  → thousands of cases

The testing library generates random inputs, runs the property, and shrinks failures to minimal counterexamples.

The Three Phases of PBT

1. GENERATE  → library produces random inputs from domain
2. RUN       → property function evaluated for each input
3. SHRINK    → on failure, reduce input to smallest counterexample

Shrinking — Why It Matters

Shrink failure: [3, 9, 7, 1, 8, 2, 5] → [] (empty list triggers the bug) Shrinking transforms massive inputs into minimal, readable bugs.

When PBT Beats Unit Testing

Scenario	Why PBT wins
Boundary conditions	Hard to enumerate: negatives, zero, max_int, Unicode
Commutativity/associativity	Test f(a,b) == f(b,a) for thousands of pairs
Round-trip serialization	Encode→Decode→Decode == Encode
Invariant preservation	"sorting never loses elements"
Protocol/state machines	Exhaustively explore state transitions
Async race conditions	Generate interleavings

Language Tooling

Language	Library	Best For
Haskell	QuickCheck	The original, pure FP
Python	Hypothesis	Django, Flask, data pipelines
JavaScript	Fast-Check	Node.js, TypeScript
Go	testing/quick	Standard library, simple cases
Rust	proptest	Rust idiomatic PBT
Java	JUnit-QuickFIX	Legacy Java systems

---

Python + Hypothesis

Setup

pip install hypothesis pytest

Basic Pattern

from hypothesis import given, strategies as st, assume, settings

@given(st.lists(st.integers(min_value=1, max_value=1000), min_size=1))
def test_sort_preserves_elements(xs):
    """Sorting never loses or duplicates elements."""
    sorted_xs = sorted(xs)
    assert len(sorted_xs) == len(xs)
    assert set(sorted_xs) == set(xs)          # preservation
    assert sorted_xs == sorted(sorted_xs)      # idempotence

@given(st.lists(st.integers()), st.lists(st.integers()))
def test_merge_preserves_all(a, b):
    """Merging two lists preserves all elements."""
    merged = sorted(a + b)
    expected = sorted(sorted(a) + sorted(b))
    assert merged == expected