interval-guided-regression-test-update

Interval-Guided Regression Test Update

Automatically update regression tests to maintain interval coverage when program code changes.

Core Concept

Intervals represent ranges of values that variables can take during execution. When code changes, intervals may be added, removed, or modified. This skill ensures regression tests continue to cover all important intervals.

Example:

# Old code
def process(x):
    if x < 10:
        return x * 2
    else:
        return x + 10

# Intervals: [0, 10), [10, ∞)

# New code (added negative handling)
def process(x):
    if x < 0:
        return -x
    elif x < 10:
        return x * 2
    else:
        return x + 10

# Intervals: (-∞, 0), [0, 10), [10, ∞)
# Need to add test for new interval: x < 0

Workflow

1. Analyze Existing Tests

Parse the current regression test suite:

• Identify test inputs
• Determine which intervals each test covers
• Calculate current interval coverage

# Example test analysis
test_process_small: input=5, covers [0, 10)
test_process_large: input=15, covers [10, ∞)
# Coverage: 2/2 intervals = 100%

2. Extract Interval Information

Obtain interval information from the updated program:

From static analysis:

• Parse conditionals (if, switch, etc.)
• Extract comparison operators
• Derive interval constraints

From profiling:

• Run existing tests
• Track observed value ranges
• Identify intervals exercised

From symbolic execution:

• Execute with symbolic values
• Collect path constraints
• Derive interval constraints

See references/interval-analysis.md for detailed extraction methods.

3. Identify Coverage Gaps

Compare current coverage with new intervals:

old_intervals = {[0, 10), [10, ∞)}
new_intervals = {(-∞, 0), [0, 10), [10, ∞)}

added_intervals = {(-∞, 0)}  # Need new test
removed_intervals = {}
modified_intervals = {}

4. Update Tests

Apply appropriate update strategies:

Add tests for new intervals:

# New test for (-∞, 0)
def test_process_negative():
    assert process(-5) == 5