run-test-plan

Run Test Plan

Execute a YAML test plan, run setup commands, health checks, and each test sequentially. Stop on first failure with rich debug output.

Prerequisites

• agent-browser CLI: Browser tests require the agent-browser command-line tool to be available on PATH

Arguments

• --plan <path>: Path to test plan (default: docs/testing/test-plan.yaml)
• --skip-setup: Skip setup commands and health checks (for re-running after failure)

Step 1: Parse Test Plan

Read and validate the test plan:

# Resolve plan path from --plan (default shown)
PLAN_PATH="${PLAN_PATH:-docs/testing/test-plan.yaml}"

# Check file exists
ls "$PLAN_PATH" || { echo "Error: Test plan not found: $PLAN_PATH"; exit 1; }

# Validate YAML
python3 -c "import yaml; yaml.safe_load(open('$PLAN_PATH'))" || { echo "Error: Invalid YAML: $PLAN_PATH"; exit 1; }

Extract from the YAML:

• setup.commands: List of setup commands
• setup.health_checks: List of URLs to poll
• tests: Array of test cases

Step 2: Run Setup (unless --skip-setup)

2a. Check Prerequisites

If setup.prerequisites exists, verify each one:

# For each prerequisite in setup.prerequisites
<prerequisite.check> || { echo "Prerequisite not met: <prerequisite.name>"; exit 1; }

2b. Set Environment Variables

If setup.env exists, export each variable. Variables using ${VAR} syntax should be resolved from the current environment:

# For each key/value in setup.env
export <key>="<value>"

2c. Build

If setup.build exists, execute build commands sequentially:

# For each command in setup.build
<command> || { echo "Build failed: <command>"; exit 1; }

2d. Start Services

If setup.services exists, start long-running processes and wait for health checks:

# For each service in setup.services
nohup <service.command> > .beagle/service-<index>.log 2>&1 &
echo $! > .beagle/service-<index>.pid

For each service with a health_check, poll until ready: