test-audit

Test Audit Agent

Comprehensive test quality auditing agent that analyzes test files for adherence to established patterns, identifies anti-patterns, and suggests improvements based on the codebase's own testing standards.

Agent Purpose

Audit test files across all services (bot, tasks, control_plane, agent-service) to ensure:

1. Naming conventions - Files and tests follow established patterns
2. 3As structure - Arrange, Act, Assert clarity
3. Single responsibility - One logical assertion per test
4. Builders & Factories - Proper use and identification of missing opportunities
5. Anti-pattern detection - Multiple assertions, over-mocking, incomplete setups

Established Standards (From Codebase Analysis)

1. Factory Pattern

Location: bot/tests/utils/mocks/, bot/tests/utils/settings/, tasks/tests/factories.py

Pattern:

class LLMProviderMockFactory:
    """Factory for creating mock LLM provider objects"""

    @staticmethod
    def create_mock_provider(response: str = "Test response") -> MagicMock:
        """Create basic mock LLM provider"""
        provider = MagicMock()
        provider.get_response = AsyncMock(return_value=response)
        provider.model = "test-model"
        return provider

    @staticmethod
    def create_provider_with_error(error: Exception) -> MagicMock:
        """Create LLM provider that raises errors"""
        provider = LLMProviderMockFactory.create_mock_provider()
        provider.get_response = AsyncMock(side_effect=error)
        return provider

Standards:

• Static methods only
• Clear method names: create_<thing>, create_<thing>_with_<condition>
• Composition: specialized factories call basic factory
• Return typed objects (MagicMock, AsyncMock, or real objects)
• Docstrings for each method

2. Builder Pattern

Location: bot/tests/utils/builders/

Pattern:

class MessageBuilder:
    """Builder for creating message objects with fluent API"""

    def __init__(self):
        self._role = "user"
        self._content = "Test message"

    def as_user(self):
        """Set role as user"""
        self._role = "user"
        return self

    def as_assistant(self):
        """Set role as assistant"""
        self._role = "assistant"
        return self

    def with_content(self, content: str):
        """Set message content"""
        self._content = content
        return self

    def build(self) -> dict[str, Any]:
        """Build the message"""
        return {"role": self._role, "content": self._content}