test-integrity-gate

A check for AI-written tests that asks whether a new test actually fails when run against the old buggy code. It then checks that the test passes after the code change.

In plain words
What is it for?
Verifying bug-fix tests, checking that expected failures are meaningful, and spotting weakened assertions or skipped coverage.
Why use it?
A test can pass while failing to detect the bug it was meant to catch. This check provides evidence that the test protects against the original problem.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/zaingz/coding-quality-loop/test-integrity-gate
Any agent
npx skills add zaingz/coding-quality-loop --skill test-integrity-gate
Clone the repo
git clone --depth 1 https://github.com/zaingz/coding-quality-loop

Made for: Claude Code, Codex.

Per session 29 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 310 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00029 $0.00310
Opus 5 $0.00015 $0.00155
Sonnet 5 $0.00006 $0.00062
Haiku 4.5 $0.00003 $0.00031

Measured 2d ago against content hash 90855b2dcf9f, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

test-integrity-gate scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

The scan reads SKILL.md. This mod also ships 1 executable file (scripts/test_integrity.py), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/test-integrity-gate/SKILL.md · 36 lines

What it actually says

Test Integrity Gate

Your agent's green tests may be lying. This gate checks the important question: does the proposed test fail against the old code?

Run

From the repository root:

python3 "<skill-root>/scripts/test_integrity.py" --base <base-ref> --command "<focused test command>"

Replace <skill-root> with this skill's installed directory. Use the merge target or task-start commit as <base-ref>, not the current HEAD.

Decision

  • PASS only when the command fails against the base source with the proposed test files overlaid, then passes in the current working tree.
  • FAIL when it passes against old source, fails now, times out, or the worktree cannot be built. "Not proven" never counts as success.
  • Separately inspect the diff for edited expected values, weakened matchers, removed assertions, skips, or lower coverage thresholds. Those are review clues; replay is the blocking proof.

For completion contracts, independent review, diff scope, and CI receipts, use the full Coding Quality Loop.

Files

What ships with it

4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 36 lines · 29 tokens per session scan A 90855b2dcf9f

Subscribe to this mod's changes

test-integrity-gate is a skill published in the GitHub repository zaingz/coding-quality-loop (5 stars, last pushed 23d ago), licensed MIT. It adds 29 tokens to every session and 310 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

arxiv-paper-writer

Write LaTeX ML/AI review articles for arXiv using the IEEEtran template and verified BibTeX citations. Use when writing, planning, or continuing an arXiv review/survey paper, or when validating and repairing citations in an existing LaTeX project.

appautomaton/latex-arxiv-SKILL · 62 tokens

collaborating-with-gemini

Use the Gemini CLI to consult Gemini and delegate coding tasks for prototyping, debugging, and code review. Supports multi-turn sessions via SESSIONID. Optimized for low-token, file/line-based handoff.

appautomaton/latex-arxiv-SKILL · 50 tokens

skills-constitution

当 Agent 接到专业任务(编码/爬虫/文件操作/API调用/数据分析/文档/部署/推送等)时,强制先查记忆层和技能索引,有匹配必用、无匹配必搜、答复时自动推荐(排除已装)。用于防止 Agent 跳过技能直接硬扛通用能力。跨平台通用(WorkBuddy/Claude/ChatGPT/Cursor/Gemini 等 20+ 框架)。完整版本史见 CHANGELOG.md。.

jiabaobei/skills-constitution · 119 tokens

latex-rhythm-refiner

Post-process LaTeX project prose to improve readability through varied sentence and paragraph lengths. Removes filler phrases and unnecessary transitions while preserving all citations and semantic meaning. Use after drafting is complete, when sections read as monotonous or blocky.

appautomaton/latex-arxiv-SKILL · 53 tokens

collaborating-with-claude

Use the Claude Code CLI to consult Claude and delegate coding tasks for prototyping, debugging, and code review. Supports multi-turn sessions via SESSIONID. Optimized for low-token, file/line-based handoff.

appautomaton/latex-arxiv-SKILL · 51 tokens

zod

Zod v4 best practices, patterns, and API guidance for schema validation, parsing, error handling, and type inference in TypeScript applications. Covers safeParse, object composition, refinements, transforms, codecs, branded types, v3→v4 migration, and testing schemas with Jest or Vitest. Baseline: zod ^4.3.0. Triggers…

anivar/zod-skill · 138 tokens