fetch-test-runs

Fetch Test Runs

This skill fetches test runs from the Sippy API. It can return both failed and successful test runs, including JUnit output for AI-based analysis.

When to Use This Skill

Use this skill when you need to:

• Fetch test run data for a specific test across all jobs
• Get raw test failure outputs for AI-based similarity analysis
• Compare error messages across runs to determine if they share the same root cause
• Include successful runs in addition to failures (optional)
• Filter runs by job name substrings (e.g., only GCP techpreview jobs)
• Access JUnit test output for debugging and investigation

Prerequisites

1. Network Access: Must be able to reach the Sippy test runs API

   • No authentication required
   • Check: curl -s https://sippy.dptools.openshift.org/api/tests/v2/runs?test_id=test

2. Python 3: Python 3.6 or later

   • Check: python3 --version
   • Uses only standard library (no external dependencies)

3. Input Data: Requires test_id (job_run_ids are optional)

   • Get from fetch-regression-details skill output
   • test_id: Found in regression data (e.g., "openshift-tests:71c053c318c11cfc47717b9cf711c326")
   • job_run_ids: Optional - extracted from sample_failed_jobs[].failed_runs[].job_run_id

Implementation Steps

Step 1: Run the Python Script

# Path to the Python script
script_path="plugins/ci/skills/fetch-test-runs/fetch_test_runs.py"

# Fetch all test runs (failures only, by default)
python3 "$script_path" "$test_id" --format json

# Include successful runs as well
python3 "$script_path" "$test_id" --include-success --format json

# Filter to a specific Prow job (exact name works as substring of itself)
python3 "$script_path" "$test_id" --job-contains "periodic-ci-openshift-release-..." --format json

# Filter by multiple substrings (AND logic, case-insensitive, server-side)
python3 "$script_path" "$test_id" --job-contains gcp --job-contains techpreview --format json

# Filter to specific job run IDs (backward compatible with analyze-regression)
python3 "$script_path" "$test_id" "$job_run_ids" --format json

# Get human-readable summary
python3 "$script_path" "$test_id" --format summary