mainframe-test-audit

Test-system audit

Apply this method when the active work matches the description, whether you are the primary auditor or a delegated specialist. Follow the scope and authority supplied through the current execution path; this skill does not expand either.

Audit the existing test system rather than redesigning it from preference. Tests protect observable guarantees, not a target count, coverage percentage, framework fashion, or universal ratio of test levels.

Establish the audit boundary

Identify the exact package, component, journey, test layer, suite, or execution contour under review. State the observable product guarantees inside that boundary and the concrete regressions the tests are expected to detect.

Inspect the effective project instructions, relevant product decisions and contracts, owning code paths, test configuration, existing tests, fixtures and lifecycle setup, native commands, and applicable CI configuration. Do not broaden a bounded audit into a repository-wide review.

Map how the project obtains each kind of evidence:

• focused checks used while changing one behavior;
• the nearest fast local regression suite;
• checks whose correctness depends on a real local dependency;
• broad, compatibility, deployed, or full-system checks reserved for CI or a deliberately prepared environment.

The absence of one universal test command is not a defect. Judge whether the applicable checks can be discovered, understood, and rerun without unrelated infrastructure or hidden side effects.

Test the quality of the evidence

For every material guarantee, identify the cheapest observation that would faithfully fail for the real regression. Inspect whether the current evidence:

• reaches the behavior through the boundary where the defect can occur;
• asserts an observable contract strongly enough to detect the failure;
• uses a mock, stub, emulator, database substitute, or fixture that preserves the relevant semantics;
• avoids coupling to private implementation details that can change without changing behavior;
• covers only branches that actually exist, without duplicating the same guarantee for ceremony;
• isolates time, randomness, ordering, process state, shared data, network access, and concurrency sufficiently to be repeatable;
• makes setup, infrastructure, runtime, and maintenance cost visible and proportional to the risk.