off-by-none

Spec-Faithful Implementation with Independent Verification

Overview

Self-written tests inherit the same misreading as your implementation, so they pass while the code is wrong. This skill forces test cases to be derived from the spec's literal wording BEFORE implementing, with mandatory boundary-value coverage.

Core principle: a test derived from your implementation verifies nothing. Derive every test from the spec's exact words.

Workflow

1. Requirements ledger. Number every clause of the spec, including sub-clauses hidden inside sentences ("then re-apply rule 6" is its own entry). One checklist item per clause.

2. Derive tests from the spec text — before writing any implementation code.

   • For every clause, write at least one case using only the spec's wording, not your planned code.
   • Boundary rule (mandatory): every numeric limit or boundary word — "at", "at or before", "exceeds", "up to", "at least", "within" — gets three cases: exactly N-1, exactly N, exactly N+1. Compute the expected outputs by hand from the spec sentence, character by character if needed.
   • Boundary words bind to what they modify. "At or before max_len" about a position (a delimiter, a marker, a match) requires a test where that thing sits exactly AT position max_len — a test where only the total length is at the limit does not cover it. For each boundary phrase, name the noun it modifies (length? index? count? position?) and put the test exactly there.
   • For every "if/when/unless" clause, one case per branch.

3. Implement.

4. Run the tests. Paste actual output. "I verified" without shown output is not verification.

5. Adversarial re-read. Go back to the spec, clause by clause, and for each ask: "is there an input where another literal reading of this sentence gives a different answer than my code?" Ambiguity between two readings = write a test for both readings and pick the one the spec's words actually say.

6. Claim done only with evidence. The final report has three REQUIRED sections (write "none" explicitly when empty):

   • Verified: the ledger with every item checked + pasted test output.
   • Assumed: every spec reference you could not inspect — external systems, config values, "same as X", "matching the existing behavior of Y". One line each. An unverifiable reference makes the ship claim conditional: state what must be confirmed, never an unqualified "ready to ship".
   • Seen but not touched: anything broken or suspicious in the code you worked around, outside the requested change. Before writing this section, re-read every function in every file you edited, including the ones you didn't change. For each untouched function, state what it literally does to its input step by step (especially any transformation that happens before a check) — observations only. Never write a verdict like "works correctly" on code you did not spec-test; you have no evidence for it. Report; do not fix. "None" means "I read the surrounding code and found nothing", never "I didn't look". "Minimal change" limits the diff, not the report.