Getting it into your agent
It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.
git clone --depth 1 https://github.com/PramodDutta/qaskillsnpx agentmods add skills/pramoddutta/qaskills/flaky-test-quarantineWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/pramoddutta/qaskills/flaky-test-quarantine)<a href="https://agentmods.dev/skills/pramoddutta/qaskills/flaky-test-quarantine"><img src="https://agentmods.dev/badge/skills/pramoddutta/qaskills/flaky-test-quarantine/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/pramoddutta/qaskills/flaky-test-quarantine"><img src="https://agentmods.dev/badge/skills/pramoddutta/qaskills/flaky-test-quarantine.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 13 findings, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium MCP Rug Pull · line 97 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 218 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 220 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 831 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 835 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 867 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 871 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 916 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 930 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 1064 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 238 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 877 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 883 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00032 | $0.09032 |
| Opus 5 | $0.00016 | $0.04516 |
| Sonnet 5 | $0.00006 | $0.01806 |
| Haiku 4.5 | $0.00003 | $0.00903 |
Grade A, and why
Flaky Test Quarantine scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Runs shell commandslowCapability
Expected in a hook, worth knowing in a rule or an instructions file.
import { execSync } from 'child_process'; How it starts
The opening of the file, as written. The whole thing — 1,071 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Flaky Test Quarantine
Flaky tests are tests that produce different outcomes (pass or fail) when run against the same code under the same conditions. They erode confidence in the test suite, train developers to ignore failures, and slow down CI pipelines with unnecessary retries. A single flaky test in a suite of 500 can cause the entire pipeline to require re-runs, wasting developer time and compute resources. This skill provides a systematic approach to detecting, quarantining, categorizing, and fixing flaky tests while maintaining CI pipeline stability.
Core Principles
-
Detection Before Quarantine: A test must be proven flaky through repeated execution before it is quarantined. A single failure does not make a test flaky; it might indicate a real bug. Multi-run analysis with statistical tracking separates genuine flakiness from legitimate failures.
-
Quarantine Is Not Deletion: Quarantined tests must remain in the codebase and continue running in a separate pipeline. Quarantine is a temporary holding pattern that prevents flaky tests from blocking the main pipeline while preserving the test and tracking its behavior.
-
Root Cause Categorization Drives Fixes: Different types of flakiness require different fixing strategies. Timing issues need explicit waits, state leakage needs proper cleanup, external dependencies need mocking, and race conditions need synchronization. Categorizing the root cause directs the fix.
-
Flakiness Is Measurable: Track the flakiness rate (failures per total runs) for every test over time. This metric determines quarantine decisions, fix prioritization, and verifies that fixes actually resolved the issue.
-
Zero-Tolerance Pipeline: The main CI pipeline must be green to merge. Allowing occasional failures or manual re-runs normalizes flakiness and makes it impossible to distinguish real failures from known flaky ones.
-
Fix the Root Cause, Not the Symptom: Adding retries or increasing timeouts masks flakiness without fixing it. These approaches hide real issues and increase overall test execution time. Address the underlying non-determinism.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 6d ago First seen · 1,071 lines · 32 tokens per session scan A 5572c67090e5
Flaky Test Quarantine is a skill published in the GitHub repository PramodDutta/qaskills (220 stars, last pushed 10d ago), licensed MIT. It adds 32 tokens to every session and 9,032 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 1 finding (runs shell commands). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
maas-nightly-qe-impact
Assess whether a Models-as-a-Service autofix change requires follow-up in opendatahub-tests or ods-ci nightly QE pipelines. Runs as a Jira autofix postreview extension for the Model as a Service component. Appends a Nightly QE Impact section to the PR description and writes informational findings. Use when autofix…
test-case-to-katalon-studio
Convert Katalon True Platform/TestOps manual test cases into Katalon Studio automation inside a local Studio Test Project checkout. Use when you need to author or extend a .tc test case file and its paired Groovy script under Scripts/, keep test case variable GUIDs consistent with the .ts test suite bindings that read…
test-data
Design, source, seed, and tear down the test data a Katalon True Platform test case or an automated suite runs on. Use when the steps are already settled and the blocker is the values, for example which data classes a case needs, which records must exist before a run, how to keep literals out of the step text and into…
test-estimation
Estimate testing effort, duration, and resourcing for a Katalon True Platform/TestOps cycle. Use when the question is how long testing will take, how many testers it needs, whether the scope fits the sprint window, or what a scope change costs in person-hours. Sizes design, manual execution, automated execution and…
test-reporting
Report Katalon True Platform/TestOps quality metrics to people outside QA. Use when you need to answer a stakeholder question with testing data, choose the few metrics that actually answer it, trend coverage, execution health, defect risk and stability across several releases, sprints, or iterations rather than inside…
true-platform-testing
End-to-end Katalon True Platform testing workflow and lifecycle router. Use when one request spans several stages and no single skill owns all of it, for example analyze a requirement, design and import the cases, build a suite, run it with AI, and report the outcome. Also use to route any testing request across the…