Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add fishzjp/qa-skills --skill test-reliabilitygit clone --depth 1 https://github.com/fishzjp/qa-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/fishzjp/qa-skills/test-reliability)<a href="https://agentmods.dev/skills/fishzjp/qa-skills/test-reliability"><img src="https://agentmods.dev/badge/skills/fishzjp/qa-skills/test-reliability/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/fishzjp/qa-skills/test-reliability"><img src="https://agentmods.dev/badge/skills/fishzjp/qa-skills/test-reliability.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00122 | $0.03316 |
| Opus 5.5 | $0.00049 | $0.01326 |
| Sonnet 5.5 | $0.00024 | $0.00663 |
| Haiku 4.5 | $0.00012 | $0.00332 |
Grade A, and why
test-reliability scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 116 lines — stays where its author put it; the contents beside it link to each section on GitHub.
测试可靠性治理（test-reliability）
测试套件自身的稳定性治理——回答一个问题：这套测试的结果还能不能信。单条用例时好时坏、重跑变绿、夜间全量靠重试撑绿、发布卡点被 flaky 疲劳轰炸，都属于本 skill 的治理对象。本 skill 是套件级的"判定 → 隔离 → 根因 → 门禁 → 健康度"闭环，不替代任何单次执行流程。
- 输入：失败/翻灯历史（CI 运行记录、重跑日志、triage 分流表 D 类条目）、套件清单与运行统计；项目存在
.qa/时先读其 flaky-tests 主题（经qa-memory沉淀的历史判定） - 输出（落盘）：《可靠性报告_{套件}_{日期}.md》——逐条 flaky 清单（判定 × 根因分类 × 处置 × 状态，缺一字段即报告不合规）+ 套件健康度机读摘要（字段规范见
references/flaky-playbook.md第 5 节，挂靠../core/report-template.md机读段）；沉淀判定结论（哪些发现值得写入.qa/，交qa-memory工作流执行） - 边界（三层分工，同源
../core/triage.md第 7 节）：执行中单条失败的即时定性（first-run-flaky 规则、等待策略）→ 各执行 skill 工程约定；一批失败的首轮分类与路由 →../core/triage.md；已确认 Bug 的根因/影响/修复建议 →bug-analysis；本 skill 管"套件与门禁"层——单条定性做完之后的跨轮追踪、根因分类、隔离策略、重试语义与健康度
When to Use
- 用户问"这条用例为什么时好时坏 / 这算不算 flaky / 重跑绿了算修好吗"
- CI 夜间套件靠重试撑绿，需要裁决重试策略或制定隔离（quarantine）门禁
- 需要产出套件健康度（flaky 率、隔离数、重试救回率）或发布卡点的可靠性证据
- 分流表 D 类（不稳定）条目跨轮累积，需要根因分类与治理排期
- 项目要给"测试套件可信度"立规矩：什么样的套件状态允许作为发布证据
When NOT to Use
- 一次执行中的单条失败即时处理（截图、重跑定性、等待降级）→ 各执行 skill 工程约定（如
automated-e2e-testing） - 一轮执行结束失败 ≥3 条的批量定性分流 →
../core/triage.md（其结论为 D 类的条目跨轮追踪时回到本 skill） - 已确认 Bug 的根因定位、影响分析与修复建议 →
bug-analysis - 写测试用例本身 →
test-case-writing；回归范围决策 →regression-testing - 单次执行报告 →
../core/report-template.md（本 skill 的健康度摘要挂靠其机读段）
核心模型：稳定性的三层语义
- 用例级：一条用例的"真失败 vs 噪声"判定——证据是复跑矩阵，不是感觉（工作流一）。
- 套件级：一组用例的可靠性水位——健康度指标说话（工作流五）。
- 门禁级：套件结果作为发布证据的语义——重试绿、隔离跳过、未跑各算什么（工作流四）。
三层不可混谈：单条判定正确 ≠ 套件健康；套件全绿 ≠ 门禁证据合格（可能全靠重试撑着）。
工作流一：flaky 判定与根因四分类
- 判定（先于一切归因）：一条用例首次失败、原样重跑通过 → 判 first-run-flaky，不算稳定通过——重跑只用于定性，不得成为常态化通过手段（同源：playwright-conventions §11、
../core/triage.mdD 类，规则一致）。证据不足时显式标"未知"，不许硬编结论。 - 复跑矩阵（根因归因的证据面）：对已定性 flaky 的用例按矩阵取证——原样重跑 / 隔离重跑（单跑该条）/ 干净环境重跑（新容器/新目录）/ 带序重跑（按序执行邻居）。矩阵结果 → 根因四分类的判定树此时加载
references/flaky-playbook.md（第 2–3 节：判定树 + 证据表）。 - 根因四分类（类名 × 典型证据见 playbook 第 3 节，修复模式见 playbook 第 4 节）：
- S1 时序与等待不足：异步未等、轮询窗口、动画/懒加载——隔离重跑仍间歇失败
- S2 共享状态与测试间依赖：数据残留、执行顺序敏感、全局单例污染——隔离重跑稳定、带序重跑失败
- S3 环境与第三方抖动：依赖服务超时、证书/账号过期、发布窗口重叠、资源容量——干净环境重跑消失，或失败聚集在特定时段
- S4 并发与资源竞态：并行执行下的端口/文件/数据库冲突——关并行后消失
- 产出：每条 flaky 一行记录——
用例 × 判定 × 根因类 × 证据（复跑矩阵结果）× 处置 × 状态。六字段缺一即报告不合规（可执行性纪律：不可判定的条目没有治理价值）。
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago Changed f73a04199af5
- 28d ago Changed fa20d111da61
- 1mo ago First seen · 116 lines · 122 tokens per session scan A c9134a37f5fd
test-reliability is a skill published in the GitHub repository fishzjp/qa-skills (48 stars, last pushed 9d ago), licensed MIT. It adds 122 tokens to every session and 3,316 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-09.
Other skills, from other repositories
qa-expert
Expert-level quality assurance, testing strategies, automation, and QA processes. Use when the user mentions testing, test automation, quality assurance, or Selenium, or when the task involves Testing Types, QA Processes, Test Strategy, or Defect Management.
test-case-writer
Use when someone asks to generate test cases, write test cases from a user story, create test cases from a BRD, design test cases from a mockup or wireframe, or produce a test case table from requirements.
qa-engineer
QA specialist agent for test planning, execution, and regression analysis.
test-review
A review checklist for newly written or changed automated tests using TypeScript and Playwright. Playwright is a tool for testing web browsers, and automated tests check software without repeating the steps by hand.
playwright-expert
Expert in Playwright E2E testing framework, auto-waiting mechanisms, test generation, trace viewer, and CI/CD integration. Use when the user mentions testing, end-to-end tests, QA, automation, end-to-end testing, or test automation, or when the task involves Playwright Framework, Test Organization, Advanced Features…
selenium-expert
Expert in Selenium WebDriver, Selenium Grid, page object model, waits, cross-browser testing, and test automation frameworks. Use when the user mentions testing, end-to-end tests, QA, automation, WebDriver, or Selenium grid, or when the task involves Selenium Components, Browser Support, Advanced Features, or Basic…