Prow Job Analyze Test Failure

Prow Job Analyze Test Failure

This skill analyzes test failures by downloading Prow CI artifacts, checking test logs, inspecting resources and events, analyzing test source code, and optionally integrating cluster diagnostics from must-gather data.

Prerequisites

Identical with "Prow Job Analyze Resource" skill.

Input Format

The user will provide:

1. Prow job URL - gcsweb URL containing test-platform-results/

   • Example: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_hypershift/6731/pull-ci-openshift-hypershift-main-e2e-aws/1962527613477982208
   • URL may or may not have trailing slash

2. Test name (optional) - specific test name that failed

   • When provided, focus the analysis on that test's stack trace and logs
   • When omitted, analyze all failed CI steps by inspecting the JUnit XML, build logs, and step artifacts to identify the root cause — this is the common case for multi-step CI workflows (e.g. HyperShift, bare-metal OADP jobs) where the failure is in a step rather than a named unit test
   • Examples:
     • TestKarpenter/EnsureHostedCluster/ValidateMetricsAreExposed
     • TestCreateClusterCustomConfig
     • The openshift-console downloads pods [apigroup:console.openshift.io] should be scheduled on different nodes

3. Optional flags (optional):

   • --fast - Skip must-gather extraction and analysis; continue with log/JUnit/step-artifact analysis only (scope is preserved: test-level when test-name is provided, step-level across all failed CI steps when omitted)

Implementation Steps

Step 1: Parse and Validate URL

Use the "Parse and Validate URL" steps from "Prow Job Analyze Resource" skill

Step 2: Create Working Directory

1. Check for existing artifacts first

   • Check if .work/prow-job-analyze-test-failure/{build_id}/logs/ directory exists and has content
   • If it exists with content:
     • Use AskUserQuestion tool to ask:
       • Question: "Artifacts already exist for build {build_id}. Would you like to use the existing download or re-download?"
       • Options:
         • "Use existing" - Skip to step Analyze Test Failure
         • "Re-download" - Continue to clean and re-download
     • If user chooses "Re-download":
       • Remove all existing content: rm -rf .work/prow-job-analyze-test-failure/{build_id}/logs/
       • Also remove tmp directory: rm -rf .work/prow-job-analyze-test-failure/{build_id}/tmp/
       • This ensures clean state before downloading new content
     • If user chooses "Use existing":
       • Skip directly to Step 4 (Analyze Test Failure)
       • Still need to download prowjob.json if it doesn't exist