test-reporting

Integration Test Reporting Skill

Run the Level 2 dummy agent integration test suite and produce a detailed HTML report with per-test input → outcome analysis.

Trigger

User wants to run integration tests and see results:

• /test-reporting
• /test-reporting test_component_queen_live.py
• /test-reporting --all

SOP: Running Tests

Step 1: Select Scope

If the user provides a specific test file or pattern, use it. Otherwise run the full suite.

# Full suite
cd core && echo "1" | uv run python tests/dummy_agents/run_all.py --interactive 2>&1

# Specific file (requires manual provider setup)
cd core && uv run python -c "
import sys
sys.path.insert(0, '.')
from tests.dummy_agents.run_all import detect_available
from tests.dummy_agents.conftest import set_llm_selection

avail = detect_available()
claude = [p for p in avail if 'Claude Code' in p['name']]
if not claude:
    avail_names = [p['name'] for p in avail]
    raise RuntimeError(f'No Claude Code subscription. Available: {avail_names}')
provider = claude[0]
set_llm_selection(
    model=provider['model'],
    api_key=provider['api_key'],
    extra_headers=provider.get('extra_headers'),
    api_base=provider.get('api_base'),
)

import pytest
sys.exit(pytest.main([
    'tests/dummy_agents/TEST_FILE_HERE',
    '-v', '--override-ini=asyncio_mode=auto', '--no-header', '--tb=long',
    '--log-cli-level=WARNING', '--junitxml=/tmp/hive_test_results.xml',
]))
"

Step 2: Collect Results

After the test run completes, collect:

1. JUnit XML from --junitxml output (if available)
2. stdout/stderr from the run
3. Summary table from run_all.py output (the Unicode table)

Step 3: Generate HTML Report

Write the report to /tmp/hive_integration_test_report.html.

The report MUST include these sections:

Header

• Run timestamp (ISO 8601)
• Provider used (model name, source)
• Total tests / passed / failed / skipped
• Total wall-clock time
• Overall verdict: PASS (all green) or FAIL (with count)