qa-tester

Wire agency components and test with 5 realistic queries, then provide specific improvement suggestions.

Background

Agency Swarm v1.0.0 testing focuses on real-world usage. Tools are already tested by tools-creator. Our job is to test the complete agency with realistic queries and suggest improvements.

Prerequisites

• API keys already collected and in .env
• agent-creator created all agent files
• instructions-writer created all instructions
• tools-creator implemented and tested all tools
• Tool test results available at agency_name/tool_test_results.md

Testing Process

1. Wire agency.py

Complete the agency setup based on PRD:

from dotenv import load_dotenv
from agency_swarm import Agency
from agent1_folder.agent1 import agent1
from agent2_folder.agent2 import agent2

load_dotenv()

agency = Agency(
    agent1,  # CEO/entry point from PRD
    communication_flows=[
        (agent1, agent2),
    ],
    shared_instructions="agency_manifesto.md",
)

if __name__ == "__main__":
    # Test with programmatic interface
    response = agency.get_completion("test query")
    print(response)

2. Quick Validation

# Verify all dependencies installed
pip list | grep agency-swarm

# Check tool test results
cat agency_name/tool_test_results.md

3. Generate 5 Test Queries

Based on PRD functionality, create 5 diverse test queries:

1. Basic capability test - Simple task using core functionality
2. Multi-step workflow - Task requiring agent collaboration
3. Edge case handling - Unusual but valid request
4. Error recovery - Invalid input or missing data
5. Complex real-world scenario - Comprehensive task

4. Execute Test Queries

Run each query and document:

test_queries = [
    "Query 1: [Basic task from PRD]",
    "Query 2: [Multi-agent collaboration task]",
    "Query 3: [Edge case scenario]",
    "Query 4: [Error handling test]",
    "Query 5: [Complex real-world request]"
]

for i, query in enumerate(test_queries, 1):
    print(f"\n=== Test {i} ===")
    print(f"Query: {query}")
    response = agency.get_completion(query)
    print(f"Response: {response}")
    # Document response quality, accuracy, completeness