quality-engineering-api-testing

Quality Engineering: API Testing & Correctness

Priority: P1 (HIGH)

Scope & Purpose

Standardizes API correctness verification across five distinct purposes (lane: api per C1):

1. Smoke: Business readiness beyond shallow health endpoints.
2. Functional: Business rules, state changes, calculations, and error invariants.
3. Contract: Schema conformance and consumer-driven expectations.
4. Integration: Multi-service coordination, real vs mocked dependencies, retries, and idempotency.
5. Regression Selection: Evidence-based test selection with explicit baseline and uncertainty handling.

Core Rules

1. Smoke Readiness (AC-06): A 200 OK on GET /health proves process liveness, not business readiness. Smoke suites must exercise at least one end-to-end critical business operation (e.g., authenticated preflight, read core entity).
2. Functional Oracles (AC-07): Schema-valid payloads with 200 OK do not prove correctness. Every functional test must assert on computed values, database state, business invariants, and specific error payloads.
3. Contract Verification (AC-08): Do not equate OpenAPI schema validation with version compatibility or consumer contract testing. Schema checks prove candidate syntax conformance; baseline-to-candidate diffs prove what baseline clients and consumers still get. Record producer versions and the protected consumer version, then verify both directions: baseline-valid requests remain accepted (a schema default never exempts a newly required property unless the provider's actual omission behavior is demonstrated), and candidate responses stay readable by baseline consumers, including new enum/union values an exhaustive consumer rejects. Consumer-driven contracts (e.g., Pact) verify consumer-needed fields and semantics. Missing baseline or consumer artifacts are BLOCKED, never inferred compatible.
4. Integration & Idempotency (AC-09): Firing network requests without verifying resulting persistence or side effects is invalid. When testing retries (including post-effect lost-response timeouts), verify final state and assert zero duplicate side effects across both local database records and an independent downstream gateway ledger oracle. Simulated mocks do not replace authorized real-dependency evidence.
5. Regression Selection (AC-10): Map code/contract deltas to affected obligations. Always retain the critical smoke baseline. If delta impact is ambiguous, broaden coverage or log a blocker—never claim unverified completeness.