test-oracle-generator

Test Oracle Generator

Generate automated test oracles to verify correct software behavior across multiple oracle types.

Oracle Types

1. Assertion-based: Compare actual vs expected output
2. Property-based: Verify invariants that should always hold
3. Differential: Compare new implementation against reference
4. Metamorphic: Test input-output transformations

Workflow

Step 1: Analyze the Function Under Test

Understand what the function does and its expected behavior.

Checklist:

• Read function signature and docstring
• Identify input parameters and types
• Identify return type and possible values
• Note preconditions and postconditions
• Check for edge cases (empty input, null, boundary values)

Example Analysis:

# Python
def calculate_discount(price: float, discount_percent: int) -> float:
    """Calculate discounted price.

    Args:
        price: Original price (must be positive)
        discount_percent: Discount percentage (0-100)

    Returns:
        Discounted price
    """
    return price * (1 - discount_percent / 100)

Analysis:

• Inputs: price (float, must be > 0), discount_percent (int, 0-100)
• Output: float (discounted price)
• Invariants: Result should be ≤ original price, result should be ≥ 0
• Edge cases: 0% discount, 100% discount, boundary values

Step 2: Generate Assertion-Based Oracles

Create explicit expected value assertions for common cases.

Template:

# Python (pytest)
def test_<function>_<scenario>():
    # Arrange
    input1 = <value>
    input2 = <value>
    expected = <calculated_expected_value>

    # Act
    actual = function_under_test(input1, input2)

    # Assert
    assert actual == expected, f"Expected {expected}, got {actual}"

// Java (JUnit)
@Test
public void test<Function><Scenario>() {
    // Arrange
    Type input1 = <value>;
    Type input2 = <value>;
    Type expected = <calculated_expected_value>;

    // Act
    Type actual = functionUnderTest(input1, input2);

    // Assert
    assertEquals(expected, actual, "Expected and actual should match");
}