testing-strategy

Testing Strategy

Tests are the only correctness instrument you control at build time: shape the suite to the code, not to a decades-old ratio, and make each test assert behaviour a build either passes or fails. Applies to every test under test/ and integration_test/.

Read the reference for the task at hand:

• references/test-layers.md — per-layer harness, imports, teardown, edge tables (pure core, in-memory Drift, headless Notifier, integration).
• references/property-and-fakes.md — fakes-over-mocks, enum-driven fake state, seeded fuzz, independent oracles, absence-of-a-failure-class tests.
• references/coverage-and-budget.md — file-level floors, the coverage-lies-upward fix, the suite-time budget, and the manual-pass handoff.

Run scripts/check_test_hygiene.sh and scripts/run_tests.sh before a PR.

Golden, RTL, and a11y widget mechanics live in widget-golden-and-a11y-testing; this skill governs everything below the pixel.

Non-negotiable rules

1. Shape the suite to the code, not the pyramid. The 70/20/10 numbers trace to a 2011 test-size heuristic (its author said they were "pulled out of a hat") and never described Flutter's unit/widget/integration taxonomy. Test at the cheapest tier that can assert the behaviour: anything expressible as f(input) -> output is a unit or property test, never a pumpWidget. Driving pure logic through the widget tree is slower, flakier, and hides which layer broke.
2. Put business rules in a Flutter-free package and inject a Clock. Domain math lives in pure Dart with zero Flutter/plugin/IO imports, tested with package:test (not flutter_test). The one time type is package:clock's Clock — NEVER DateTime.now(), never a bespoke ClockService. Pure core reads the ambient clock.now(), pinned in tests with withClock(Clock.fixed(t), …); Riverpod/feature code reads the injected clockProvider, overridden with clockProvider.overrideWithValue(Clock.fixed(t)) (seam owned by value-objects-money-and-units). The ban is structural: a pure package declares no Flutter SDK constraint.
3. Assert invariants, not just examples. Every conversion has a round-trip test (decode(encode(x)) == x) and rounding goldens at half-way/boundary values; every universal claim is a seeded fuzz loop (for (var seed = 0; seed < N; seed++)) or property test, checked against an independent oracle — never the production function under test. Print the generated input in reason: so a failure is its own minimal repro. == on integers only; closeTo(x, 1e-9) on doubles.
4. Test the data layer against a real in-memory engine. Use NativeDatabase.memory(), never a mocked DAO — a mocked DAO proves nothing about SQL, constraints, indexes, or migrations. Mock repositories only above the data layer. addTearDown(db.close) and close streams synchronously so reactive .watch() streams do not leak "Timer still pending".
5. Prefer bare-implements fakes over mocks for code you own. A class FakeX implements X (no noSuchMethod superclass) makes an interface change a compile error, models the risk (state, not call-order) as a field, and doubles as the contract's documentation. Reserve mocktail for genuinely external dependencies; never mockito/@GenerateMocks (codegen, no null-safety win).
6. registerFallbackValue for every custom type passed to any()/captureAny() in setUpAll. mocktail throws at runtime, not compile time, on a missing fallback — this is the single most common mocktail failure.
7. Drive Notifiers headlessly with ProviderContainer. Override provider dependencies via overrideWith; never pump a widget to test state. Assert on the exposed AsyncValue/state, drive actions through .notifier, and dispose the container in teardown.
8. Guard one end-to-end acceptance gate. A single realistic scenario asserted to the exact expected result, plus the conservation invariant (parts sum to the whole). It is the test that proves the pieces compose. Back it with a runtime assert tripwire inside the primitive itself (assert(sum(result) == total)) — free in release, catches the bug the moment it happens.
9. Hold an unrecoverable-bug-files floor by diff-review, not a coverage gate. There is no automated percentage gate — coverage is a published report (ci-pipeline-and-gates owns that). Instead hold a 100% floor on the handful of files (migrations, the money/allocate primitive, the parser with wire-format traps) where a gap is silent data loss — enforced by reading the diff, not by counting lines. A directory or global percentage gate rewards vanity tests and exclusion churn. First fix the coverage-lies-upward gap: flutter test --coverage omits files no test imports, so the number overstates safety.
10. Never pumpAndSettle() on an indefinite animation (splash, shimmer, spinner) — it hangs on a 10-minute timeout. Use timed pump(Duration) with fakeAsync.
11. Keep the suite fast, and hand structurally-untestable paths to a manual pass. A suite that costs minutes gets skipped, and a skipped suite is a distrusted one. Anything an emulator cannot reproduce (real audio, OEM device diversity, native surfaces without a Flutter engine) is enumerated in a named manual pre-release pass, not faked green — a green test that proves nothing is worse than an admitted gap.