validation-protocol

Fix Validation Protocol

Overview

Before claiming any fix works, follow this scientific validation protocol.

When to Use This Skill

• Claiming a bug fix is complete
• Verifying a code change works as intended
• Validating an implementation meets requirements
• Confirming a refactoring preserves functionality
• Testing that a feature behaves correctly

Core Principle

Success = Observing the intended behavior, not absence of errors.

A fix that "runs without failing" is not validated. A fix that demonstrates the specific expected outcome is validated.

The Protocol

Step 1: Reproduce the Failing State

Objective: Establish the broken baseline before attempting any fix.

Actions:

• Explicitly create or verify the broken condition exists
• Document the observable symptoms:
  • Exact error messages
  • Wrong values or outputs
  • Unexpected behavior
  • System state issues
• Confirm you can observe the failure consistently

Example:

# Create the broken state by running the relevant command or test
{test command from language manifest} -k test_broken_function

# Observe the failure
# Expected: Error or incorrect output
# Observed: Error or incorrect output confirmed

Step 2: Define Success Criteria

Objective: State what specific observable output indicates the fix worked.

Actions:

• Identify the specific behavior that proves success
• Define measurable, observable outcomes
• Distinguish success from non-success indicators:
  • Success = The fix's intended behavior is demonstrated
  • Success != Absence of errors
  • Success != Absence of warnings
  • Success != "It ran without failing"

Example:

Success Criteria:
- Function returns expected value: {"status": "processed", "count": 42}
- No exceptions raised
- Output matches test assertion exactly
- Performance within acceptable range (<100ms)

Step 3: Apply the Fix and Observe

Objective: Implement the fix and capture what actually happens.