spec-compliance-auditor

Spec Compliance Auditor

Compare the implementation against a spec file to verify every specified behavior is implemented and every implemented behavior is specified.

Input Contract

• If $ARGUMENTS contains a file path to a spec: use that file as the spec source
• Default (no $ARGUMENTS): read kb/spec.md
• Fail gracefully if neither exists: report "No spec source available" and exit

Steps

1. Extract Testable Claims

SPEC_PATH="${ARGUMENTS:-kb/spec.md}"
[ -f "$SPEC_PATH" ] || { echo "No spec at $SPEC_PATH — skipping"; exit 0; }

• Read the spec file at $SPEC_PATH in full.
• For a generated kb/spec.md, use each projected qualified identifier (<namespace>/FR-NNN, <namespace>/AC-N, <namespace>/CHECK-N) as the claim reference. Do not collapse it back to an unqualified local ID. Exclude the projection itself as evidence that the claim is implemented.
• Extract every concrete, testable claim. A testable claim is a statement that can be verified by reading code or running a test. Examples:
  • "The API returns 404 when the resource is not found"
  • "Passwords are hashed with bcrypt, cost factor 12"
  • "The retry limit is 3 attempts with exponential backoff"
• Ignore aspirational or process-oriented statements ("we aim to...", "the team will...").
• For legacy prose without qualified identifiers, number each claim legacy:S1, legacy:S2, etc. The prefix makes the weaker extraction path visible in reports.

2. Verify Each Claim

For each testable claim:

1. Locate the relevant implementation code (search for function names, types, constants mentioned in the claim).
2. Read the code and determine if the behavior matches the spec.
3. Check tests — does a test exist that verifies this specific claim? Note the test file and name.
4. Classify:

Status	Meaning
PASS	Code matches spec, test exists
UNTESTED	Code matches spec, no test covering it
DIVERGE	Code behaves differently than spec states
MISSING	No implementation found for this spec item