Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add Notysoty/openagentskills --skill flaky-test-analyzergit clone --depth 1 https://github.com/Notysoty/openagentskillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/notysoty/openagentskills/flaky-test-analyzer)<a href="https://agentmods.dev/skills/notysoty/openagentskills/flaky-test-analyzer"><img src="https://agentmods.dev/badge/skills/notysoty/openagentskills/flaky-test-analyzer.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00025 | $0.01790 |
| Opus 5 | $0.00013 | $0.00895 |
| Sonnet 5 | $0.00005 | $0.00358 |
| Haiku 4.5 | $0.00003 | $0.00179 |
Grade A, and why
Flaky Test Analyzer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 212 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Flaky Test Analyzer
What this skill does
This skill directs the agent to diagnose flaky tests — tests that sometimes pass and sometimes fail without any code changes. It examines the test code, the code under test, and the failure patterns to identify the root cause category (timing, shared state, ordering, network, randomness, etc.) and then suggests targeted fixes that make the test deterministic.
Use this when a test is unreliable in CI, when a test passes locally but fails on the CI server, or when a test fails intermittently with no obvious pattern.
How to use
Claude Code / Cline
Copy this file to .agents/skills/flaky-test-analyzer/SKILL.md in your project root.
Then ask:
- "Use the Flaky Test Analyzer skill on
tests/checkout.test.ts— it fails about 1 in 5 runs in CI." - "This test passes locally but fails in CI. Use the Flaky Test Analyzer skill to diagnose it."
Provide:
- The test file (or the specific test that's flaky)
- The failure message when it does fail
- How often it fails (every time? 1 in 10? only in CI?)
- Any observations about when it fails (after a specific other test? at a specific time of day?)
Cursor
Add the instructions below to your .cursorrules or paste them into the Cursor AI pane. Provide the test code and failure output.
Codex
Paste the test file, the failure message, and any relevant context. Ask Codex to follow the instructions below.
The Prompt / Instructions for the Agent
When asked to diagnose a flaky test, follow this process:
Step 1 — Gather information
Before analyzing, ensure you have:
- The full test code, including
beforeEach,afterEach,beforeAll, andafterAllhooks - The code under test (the function or module being tested)
- The error message when the test fails (not just "test failed")
- The failure frequency pattern (always, sometimes, CI only, etc.)
- The test framework and any relevant configuration (jest.config.js, vitest.config.ts, etc.)
If any of these are missing, ask for them.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago First seen · 212 lines · 25 tokens per session scan A f1f870b7d0e8
Flaky Test Analyzer is a skill published in the GitHub repository Notysoty/openagentskills (9 stars, last pushed 24d ago), licensed MIT. It adds 25 tokens to every session and 1,790 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
Flaky Test Doctor
Diagnose flaky test failures from Playwright reports, traces, and rerun history. Classify each failure as product, test, environment, data, or unknown with cited evidence and a proposed fix. Never auto-modifies code without opt-in.
review-ugc-render
Mandatory pre-publish review gate for a UGC video render. Transcribes the finished render's AUDIO with Whisper and word-diffs it against the approved spoken script, then gates setfinalrender — blocking a render whose generated audio mis-voices a word (e.g. the approved "human-vetted" spoken as "human witted"), drops…
api-tester
Test and document API endpoints - validate responses, check status, generate examples.
visual-qa
Runs rigorous visual QA across web, terminal, and paginated surfaces with screenshot evidence and a verdict. Use for any UI build or change, or when asked whether a page, component, or TUI looks right.
agent-browser
Automates browser interactions for web testing, form filling, screenshots, and data extraction. Use when the user needs to navigate websites, interact with web pages, fill forms, take screenshots, test web applications, or extract information from web pages.
remove-ai-slops
Removes AI-generated code smells from branch changes or an explicit file list behind regression tests. Use when the user asks to clean up, deslop, or remove AI-slop patterns from recent changes.