fetch-test-report

Fetch Test Report

This skill fetches a report for an OpenShift CI test by its full name using the Sippy tests API. It returns test metadata including the BigQuery/Component Readiness test ID, Jira component, pass rates for the current and previous reporting periods, and open bug counts. The open_bugs field counts Jira bugs that mention this test by name, which can help surface bugs that have been filed but not yet triaged in Component Readiness.

When to Use This Skill

Use this skill when you need to:

• Look up a test's Component Readiness / BigQuery test ID (test_id)
• Check current and previous pass rates for a test
• Find the Jira component associated with a test
• Determine if a test is flaking, failing, or stable
• Get run counts and failure/flake breakdowns for a test
• Check if there are open Jira bugs mentioning this test (may not yet be triaged in Component Readiness)
• See a per-variant breakdown of pass rates to identify if the test fails only in certain job types

Prerequisites

1. Network Access: The Sippy API must be accessible at https://sippy.dptools.openshift.org

   • No authentication required
   • Check: curl -s https://sippy.dptools.openshift.org/api/health?release=4.22

2. Python 3: Python 3.6 or later

   • Check: python3 --version
   • Uses only standard library (no external dependencies)

Implementation Steps

Step 1: Determine the Release

If the user did not specify a release, use the fetch-releases skill to determine the latest OCP release:

release=$(python3 plugins/ci/skills/fetch-releases/fetch_releases.py --latest)

If the user specified a release, use that directly.

Step 2: Run the Python Script

The skill uses a Python script to query the Sippy tests API:

# Path to the Python script
script_path="plugins/ci/skills/fetch-test-report/fetch_test_report.py"

# Fetch test report in JSON format (collapsed — one row for the test across all variants)
python3 "$script_path" "<test_name>" --release "$release" --format json

# Or fetch as human-readable summary
python3 "$script_path" "<test_name>" --release "$release" --format summary

# Fetch per-variant breakdown (one row per variant combo — useful for identifying
# if the test is failing only in certain job types, e.g., aws vs metal, upgrade vs new-install)
python3 "$script_path" "<test_name>" --release "$release" --no-collapse --format json