regression-root-cause-analyzer

Regression Root Cause Analyzer

Overview

Systematically investigate failing regression tests to identify root causes by analyzing code changes, error messages, test dependencies, and common failure patterns.

Workflow

1. Gather Initial Information

Collect essential details about the test failure:

Test failure details:

• Which test(s) are failing?
• Error message and stack trace
• When did tests start failing? (commit, PR, date)
• Do tests fail consistently or intermittently?
• Do tests fail locally, in CI, or both?

Quick commands to gather info:

# Run the failing test
pytest path/to/test_file.py::test_name -v

# Get recent commits
git log --oneline -10

# Check current branch and status
git status
git branch

# See what changed recently
git log --since="1 week ago" --oneline

# Check for uncommitted changes
git diff

2. Analyze Error Messages

Parse the error message and stack trace for clues. See failure-patterns.md for common patterns.

Error type indicators:

ImportError / ModuleNotFoundError:

ImportError: cannot import name 'UserService' from 'app.services'

• Likely cause: Module or class renamed/moved
• Investigation: Check recent commits to app/services/

AttributeError:

AttributeError: 'User' object has no attribute 'email_address'

• Likely cause: Property renamed or removed
• Investigation: Check User class definition changes

TypeError (arguments):

TypeError: process_data() got an unexpected keyword argument 'format'

• Likely cause: Function signature changed
• Investigation: Find process_data definition and recent changes

AssertionError:

AssertionError: assert 3 == 2

• Likely cause: Logic change or test data change
• Investigation: Understand what the assertion checks

KeyError / IndexError:

KeyError: 'status'

• Likely cause: Data structure changed
• Investigation: Check response/data format changes