flaky-test-detection-stabilization

flaky-test-detection-stabilization is a skill for Claude Code, Codex from manishiitg/coding-agent-loop. It costs 37 tokens per session (677 once invoked), scanned A, original, MIT.

A browser-test workflow for finding and stabilizing flaky tests by repeating isolated attempts and comparing evidence.

In plain words
What is it for?
It helps identify browser tests that produce inconsistent results, investigate the cause, and apply reviewed stabilization changes.
Why use it?
It helps distinguish inconsistent test behavior from real product failures and supports reviewed changes to make tests more reliable.

Skill for Claude CodeCodex

Written for no agent in particular: nothing here depends on one.

Good fit It helps identify browser tests that produce inconsistent results, investigate the cause, and apply reviewed stabilization changes.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/manishiitg/coding-agent-loop/flaky-test-detection-stabilization
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add manishiitg/coding-agent-loop --skill flaky-test-detection-stabilization
Clone the repo
git clone --depth 1 https://github.com/manishiitg/coding-agent-loop

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for flaky-test-detection-stabilization

README.md
[![agentmods](https://agentmods.dev/badge/skills/manishiitg/coding-agent-loop/flaky-test-detection-stabilization/github.svg)](https://agentmods.dev/skills/manishiitg/coding-agent-loop/flaky-test-detection-stabilization)
Your own site
<a href="https://agentmods.dev/skills/manishiitg/coding-agent-loop/flaky-test-detection-stabilization"><img src="https://agentmods.dev/badge/skills/manishiitg/coding-agent-loop/flaky-test-detection-stabilization/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for flaky-test-detection-stabilization

Your own site · 80×15
<a href="https://agentmods.dev/skills/manishiitg/coding-agent-loop/flaky-test-detection-stabilization"><img src="https://agentmods.dev/badge/skills/manishiitg/coding-agent-loop/flaky-test-detection-stabilization.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 37 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 677 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00037 $0.00677
Opus 5.5 $0.00015 $0.00271
Sonnet 5.5 $0.00007 $0.00135
Haiku 4.5 $0.00004 $0.00068

Measured 1mo ago against content hash 203dcd643736, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

flaky-test-detection-stabilization scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

playbooks/agentic-engineering-platform/browser-qa/flaky-test-detection-stabilization/SKILL.md · 52 lines

How it starts

The opening of the file, as written. The whole thing — 52 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Flaky-Test Detection and Stabilization

Outcome

Create an evidence-backed workflow that detects inconsistent browser-test behavior, separates likely test, application, and environment causes, and verifies any authorized stabilization without hiding failures.

When to use

Use when the same test and approved expectation produce intermittent outcomes. Run before self-healing when instability is suspected; use self-healing only after evidence supports an eligible test-automation defect.

Discovery and user direction

Inspect the current workflow, goals, metrics, configuration, capabilities, stores, reports, and triggers before proposing changes. Summarize reusable design and gaps, then ask focused questions for unresolved customer choices such as scope, success, approvals, thresholds, ownership, and budgets. Record the answers as customer direction. Installation alone does not approve workflow changes or execution. Default to one small-team workflow; split only for incompatible access or lifecycle boundaries.

Required inputs

Resolve the canonical test/source revision, exact build and environment, fixtures, browser profile, repetition and concurrency policy, classification thresholds, allowed stabilization paths, related cases, evidence retention, and review policy.

Plan and AgentWorks tools

Use scripted steps for repeated isolated runs, completeness checks, evidence hashing, statistics, candidate verification, and approved application. Use a message sequence for cross-attempt diagnosis. Persist verified candidates as pending review, then use a separate action route to validate the later approval and apply the exact stabilization.

Knowledge and persistence

Preserve every attempt and its video, console/network logs, trace, result, timing, and provenance. Store classifications, candidates, decisions, and stability checks in durable tables/assets. Update KB facts only after verified stabilization; canonical changes remain in test source.

Validation and reporting

Read the full file on GitHub · 52 lines

Files

What ships with it

3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 1mo ago First seen · 52 lines · 37 tokens per session scan A 203dcd643736

Subscribe to this mod's changes

flaky-test-detection-stabilization is a skill published in the GitHub repository manishiitg/coding-agent-loop (13 stars, last pushed 19d ago), licensed MIT. It adds 37 tokens to every session and 677 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-14.

Related

Other skills, from other repositories

run-dashboard-e2e-local-changes

Run Playwright E2E (tests/e2e/) against the docker/ all-in-one harness so it reflects LOCAL code changes, not a stale cached image.

BlackBeltTechnology/pi-agent-dashboard · 42 tokens

rn-testing

This skill should be used when the user asks to "write a Maestro test", "create E2E flows", "add testIDs", "run UI tests", "run E2E tests", "verify a feature works", "test my screen", "set up maestro-runner", "mock network requests", "inspect store state", "write test assertions", or needs guidance on test timing…

Lykhoyda/rn-dev-agent · 106 tokens

capturing-proof

This skill should be used when the user asks to "capture proof", "record a demo of this feature", "make a video showing it works", "record the flow for the PR", "generate a PR body", "capture screenshots for the PR", "proof-capture", or when a verified feature needs PR-ready proof artifacts (video + numbered…

Lykhoyda/rn-dev-agent · 105 tokens

test-runner

Use this skill when orchestrating agentic end-to-end tests. Resolves target + profile, dispatches the right driver(s) (playwright for web today, peekaboo for macOS (issue #381)), invokes the ux-evaluator agent (opus, read-only) against driver artifacts, reconciles findings with the open issue tracker via…

Kanevry/session-orchestrator · 124 tokens

run-action

Explicit Codex workflow: Execute a learned Maestro flow ("action") by name with optional -e KEY=VALUE parameters. Looks the flow up via packages/rn-dev-agent-core/dist/learned-actions.js (same inventory as $rn-dev-agent:list-learned-actions), then replays it via cdprunaction — auto-repair-aware orchestration with…

Lykhoyda/rn-dev-agent · 99 tokens

build-and-test

Explicit Codex workflow: Build the Expo/React Native app (local or EAS), install it, start Metro, then test a requested feature end-to-end.

Lykhoyda/rn-dev-agent · 37 tokens