voyager-bench

Benchmark Model Performance

Measure and analyze model performance on Axelera AI hardware

Use This Skill When / Not When

• Use when: the user wants to quantify a working pipeline (FPS, latency, throughput, model-variant comparisons).
• Not when: performance is a suspected fault or regression -- route to voyager-debug.
• Not when: the pipeline still needs to be built -- route to voyager-launch.

Instructions

Benchmark the specified model/pipeline: $ARGUMENTS

{{INCLUDE common/voyager-sdk-setup.md}}

{{INCLUDE common/voyager-task-integration.md}}

Step 1: Environment Setup

# Environment activation is handled by Step 0/Step 3 of the setup
# include (venv/ or axelera-env/); verify it is active
python -c "import axelera" 2>/dev/null || echo "SDK env not active"

# Verify hardware only when .voyager-runtime.json reports execute_on_device
axdevice

Step 2: Basic Benchmarking

Run benchmark on deployed model using inference with performance flags:

# Benchmark with video file (run N frames, no display, show stats)
./inference.py <model> media/traffic1_1080p.mp4 --no-display --frames 500 --show-stats

# Benchmark with SDK fake video source (synthetic frames, low I/O overhead)
./inference.py <model> fakevideo:640x480@30 --no-display --frames 1000 --show-stats

# Benchmark and save tracer data to CSV
./inference.py <model> <source> --no-display --frames 1000 --show-stats --save-tracers perf.csv

Step 3: Performance Metrics

Key metrics to measure:

• System Throughput (FPS): End-to-end pipeline performance
• Device Throughput (FPS): AIPU-only performance
• Latency (ms): Time per frame
• CPU Utilization (%): Host CPU usage

Step 4: Detailed Performance Tracing

# Enable statistics display
./inference.py <model> <source> --show-stats

# Save tracer data to CSV
./inference.py <model> <source> --save-tracers performance.csv

# Append to existing file (for comparison)
./inference.py <model> <source> --save-tracers +performance.csv