test-failure-analysis

Test Failure Analysis

Usage: /test-failure-analysis or /test-failure-analysis --deep

Purpose: Comprehensive analysis of test failures with root cause identification

---

🔬 What This Command Does

Performs deep analysis of test failures to identify root causes and suggest fixes:

1. Runs tests and captures all failures
2. Groups failures by pattern/category
3. Analyzes stack traces and error messages
4. Identifies common root causes
5. Suggests fixes with priority
6. Generates actionable report

---

📊 Analysis Workflow

Step 1: Run Tests and Capture Failures

Execute test suite with detailed output:

# Run all tests, don't stop on first failure
uv run --frozen pytest tests/ -v --tb=short --maxfail=100 2>&1 | tee /tmp/test_output.txt

# Alternative: Only run previously failed tests
uv run --frozen pytest --lf -v --tb=long 2>&1 | tee /tmp/test_failures.txt

# Count failures
FAILURES=$(grep -c "FAILED" /tmp/test_output.txt)
ERRORS=$(grep -c "ERROR" /tmp/test_output.txt)
PASSED=$(grep -c "PASSED" /tmp/test_output.txt)

echo "Results: $PASSED passed, $FAILURES failed, $ERRORS errors"

Step 2: Extract Failure Information

Parse test output to extract structured data:

# Extract failed test names
grep "FAILED" /tmp/test_output.txt | \
  sed 's/FAILED //' | \
  sed 's/ - .*//' > /tmp/failed_tests.txt

# Extract error types
grep -A 5 "FAILED\|ERROR" /tmp/test_output.txt | \
  grep -E "Error|Exception|assert" > /tmp/error_types.txt

# Extract file locations
grep "FAILED" /tmp/test_output.txt | \
  sed 's/::.*//' | \
  sort | uniq -c | \
  sort -rn > /tmp/failing_files.txt

Step 3: Categorize Failures

Group failures by error pattern:

Categories:

Assertion Failures (Logic Errors):

AssertionError: assert X == Y
AssertionError: assert result is not None

Import/Module Errors:

ImportError: cannot import name
ModuleNotFoundError: No module named

Async/Event Loop Errors:

RuntimeError: Event loop is closed
asyncio.TimeoutError
RuntimeError: This event loop is already running