Math-To-Manim tests.instructions.md

Testing Guidelines

Overview

The testing infrastructure ensures the reliability of the Math-To-Manim agent pipeline. Tests are organized into unit, integration, and end-to-end (e2e) categories.

Test Organization

tests/
├── unit/           # Test individual functions/methods in isolation
├── integration/    # Test agent interactions and pipeline segments
├── e2e/           # Test complete workflows from input to output
├── conftest.py    # Shared fixtures and configuration
└── *.py           # Top-level test files for main components

Testing Framework

Core Tools

• pytest: Primary testing framework
• pytest-asyncio: For testing async code
• pytest-cov: Code coverage reporting
• unittest.mock: Mocking API calls and external dependencies

Test Markers

Use pytest markers to categorize tests:

import pytest

@pytest.mark.unit
def test_foundation_detection():
    """Unit test - no external dependencies."""
    pass

@pytest.mark.integration
def test_agent_pipeline():
    """Integration test - tests agent interactions."""
    pass

@pytest.mark.live
def test_with_real_api():
    """Live test - requires API key and makes real API calls."""
    pass

@pytest.mark.slow
def test_full_pipeline():
    """Slow test - takes > 10 seconds to run."""
    pass

@pytest.mark.parametrize("concept,expected", [
    ("addition", True),
    ("quantum mechanics", False),
])
def test_multiple_cases(concept, expected):
    """Parameterized test - runs with multiple inputs."""
    pass

Running Tests

# Run all tests
pytest tests/ -v

# Run specific category
pytest tests/unit/ -v
pytest tests/integration/ -v

# Skip expensive live tests
pytest tests/ -v -m "not live"

# Run with coverage
pytest tests/ --cov=src --cov-report=html

# Run specific test file
pytest tests/test_prerequisite_explorer.py -v

# Run specific test function
pytest tests/test_prerequisite_explorer.py::test_foundation_detection -v

# Run interactive test runner
python tests/live_test_runner.py