What is list-unstable-tests?

A command that finds OpenShift CI tests with a pass rate below 95%. OpenShift CI is the automated testing system used for OpenShift projects, and Sippy is the service that stores and reports its test results.

Use it to search tests for a specific OpenShift version and keywords, identify regression candidates, prepare instability reports, or check quality before a release.

It quickly narrows a large set of test results to tests that fail often enough to need investigation or stabilization.

list-unstable-tests — Command for Claude Code by openshift-eng

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Clone the repo

git clone --depth 1 https://github.com/openshift-eng/ai-helpers

Made for: Claude Code.

Or install ci, the plugin that ships this one along with the rest of its 34 skills, 21 commands, 2 agents.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/commands/openshift-eng/ai-helpers/list-unstable-tests.svg)](https://agentmods.dev/commands/openshift-eng/ai-helpers/list-unstable-tests)

Your own site

<a href="https://agentmods.dev/commands/openshift-eng/ai-helpers/list-unstable-tests"><img src="https://agentmods.dev/badge/commands/openshift-eng/ai-helpers/list-unstable-tests.svg" alt="Measured on agentmods" height="20"></a>

Per session 10 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.

When invoked 1,603 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00010	$0.01603
Opus 5	$0.00005	$0.00801
Sonnet 5	$0.00002	$0.00321
Haiku 4.5	$0.00001	$0.00160

Measured 8d ago against content hash 727ccec592ec, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-08, from the pricing page.

Security

Grade A, and why

list-unstable-tests scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

Origin

Copies of this mod

1 near-identical copy found in the catalogue:

list-unstable-tests — 100% identical, 0 lines differ

plugins/ci/commands/list-unstable-tests.md · 191 lines

How it starts

The opening of the file, as written. The whole thing — 191 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Name

ci:list-unstable-tests

Synopsis

/ci:list-unstable-tests <version> <keywords> [sippy-url]

Description

The ci:list-unstable-tests command queries OpenShift CI test results from Sippy and lists all tests matching the keywords that have a pass rate below 95%. This is useful for quickly identifying unstable tests that need attention.

By default, it queries the production Sippy instance at sippy.dptools.openshift.org. You can optionally specify a different Sippy instance URL to query alternative environments (e.g., QE component readiness).

This command is useful for:

Identifying unstable tests with inconsistent pass rates
Finding regression candidates for investigation
Generating reports of unstable test cases
Prioritizing test stabilization efforts
Quality gate checks before releases

Arguments

$1 (version): OpenShift version to query (e.g., "4.21", "4.20", "4.19")
$2 (keywords): Keywords to search in test names (e.g., "olmv1", "sig-storage", "operator")
$3 (sippy-url) [optional]: Sippy instance base URL. Defaults to "sippy.dptools.openshift.org" if not provided. Examples: "qe-component-readiness.dptools.openshift.org"

Implementation

Parse Arguments
- Extract version from $1 (e.g., "4.20")
- Extract keywords from $2 (e.g., "olmv1")
- Extract Sippy URL from $3 if provided, otherwise use default "sippy.dptools.openshift.org"
- Normalize URL to extract base domain for API endpoint (strip "/sippy-ng/" suffix if present)
- Add "https://" prefix if not already present

Build Sippy API Request

Construct filter JSON for the /api/tests endpoint:

filters = {
    "items": [
        {
            "columnField": "name",
            "not": False,
            "operatorValue": "contains",
            "value": keywords
        }
    ],
    "linkOperator": "and"
}

Set query parameters:
- release: The OpenShift version
- filter: JSON-encoded filter object
- sort: "asc"
- sortField: "current_pass_percentage"

Read the full file on GitHub · 191 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

8d ago First seen · 191 lines · 10 tokens per session scan A 727ccec592ec

Subscribe to this mod's changes

list-unstable-tests is a command published in the GitHub repository openshift-eng/ai-helpers (116 stars, last pushed 3d ago), licensed Apache-2.0. It adds 10 tokens to every session and 1,603 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Testing

verify

Run repository verification using the verification-loop skill.

cloudnative-co/claude-code-starter-kit · 9 tokens

qa-changes

This skill should be used when the user asks to "QA a pull request", "test PR changes", "verify a PR works", "functionally test changes", or when an automated workflow triggers QA validation of code changes. Provides a structured methodology for setting up the environment, exercising changed behavior, and reporting…

OpenHands/extensions · 63 tokens

list-unstable-tests