validation-strategy

A step-by-step policy for deciding how much testing to do, from static checks to a full test suite. TDD here means testing in stages according to the evidence and cost needed.

In plain words
What is it for?
Use it to plan environment audits, smoke tests, single-benchmark reproductions, smaller test groups, or full CPython/CinderX validation.
Why use it?
It prevents expensive full runs from being used too early and sets limits on what conclusions each level of testing can support.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/sisibeloved/cpython-optimize-skill/validation-strategy
Any agent
npx skills add sisibeloved/cpython-optimize-skill --skill validation-strategy
Clone the repo
git clone --depth 1 https://github.com/sisibeloved/cpython-optimize-skill

Made for: Claude Code, Codex.

Per session 49 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 916 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00049 $0.00916
Opus 5 $0.00024 $0.00458
Sonnet 5 $0.00010 $0.00183
Haiku 4.5 $0.00005 $0.00092

Measured 2d ago against content hash 3035d5927dec, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

validation-strategy scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

plugins/cpython-optimize-skill/skills/validation-strategy/SKILL.md · 68 lines

What it actually says

验证策略:准确与高效

目标

把“验证”拆成可晋级的阶梯。准确来自证据门禁,高效来自成本预算、缓存复用和禁止过早全量验证。

验证阶梯

等级 名称 用途 典型产物
L0 L0 静态审计 不运行或少运行,先查环境、配置、代码、ISA、命令口径 环境表、候选路径、风险清单
L1 L1 最小功能验证 证明改动或环境没有明显错 import、smoke、单元测试、最小用例
L2 L2 单 benchmark 复现 用目标用例复现性能/崩溃/热点 单 benchmark 数据、HIR、perf、gdb
L3 L3 小集合验证 验证收益或风险是否扩散到相关用例 相关 Runtime/pyperformance 子集
L4 L4 全量验证 发布、提交、报告前的最终信心 全量 Runtime、全量 pyperformance、聚合报告

禁止默认跳到 L4。近千条 RuntimeTests 功能测试和近三小时 pyperformance 性能测试全量是晋级验证,不是调试循环默认动作。

晋级规则

  • L0 没有明确环境和命令口径时,不进入性能结论。
  • L1 没有通过时,不讨论性能收益。
  • L2 没有复现目标现象时,不实施面向该现象的优化。
  • L3 只覆盖预期受影响集合;不要把小集合结论外推到全量。
  • L4 只在准备提交、报告、扩大收益声明或用户明确要求时运行。

每次晋级前写清:

  • 当前验证等级
  • 预计耗时和资源
  • 命令、日志和产物路径
  • 失败后的下一步
  • 为什么需要更高等级

成本预算

高成本动作必须先问“是否可避免或复用”:

高成本动作 优先替代
在线安装依赖 复用 wheel/cache、镜像源、已有容器
编译 CinderX 判断改动是否影响 C/C++/JIT codegen,再增量构建
全量 Runtime 先跑相关目录、失败用例、smoke 子集
全量 pyperformance 先跑目标 benchmark、相关小集合、历史异常集合

复用缓存是默认策略:容器镜像、pip cache、build 目录、benchmark 结果和日志都应优先复查再重跑。

反问 Gate

  • 用户未明确授权 L3/L4、全量 Runtime、全量 pyperformance、在线安装依赖或 CinderX 编译时,先给预计耗时和替代方案并询问。
  • 无法判断继续等待、复用缓存、换镜像、降级验证还是中止时,必须询问。
  • 用户同时要求“最快”和“正式结论”,但证据等级不足以支撑正式结论时,询问优先级。

证据要求

准确结论至少包含:

  • 环境证据:Python、编译器、CinderX commit、容器线、关键变量
  • 命令证据:真实命令、stdout/stderr、exit status、产物路径
  • 功能证据:相关测试或最小用例
  • 性能证据:口径、baseline 含义、样本/方差或可解释的单用例数据
  • 根因证据:热点、HIR/LIR/机器码、perf/gdb 或明确排除项

没有证据时只能写“假设”,不能写“根因”。

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 68 lines · 49 tokens per session scan A 3035d5927dec

Subscribe to this mod's changes

validation-strategy is a skill published in the GitHub repository sisibeloved/cpython-optimize-skill (2 stars, last pushed 4d ago), licensed MIT. It adds 49 tokens to every session and 916 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

systematic-debugging

Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes.

obra/superpowers · 21 tokens

brainstorming

You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. Explores user intent, requirements and design before implementation.

obra/superpowers · 37 tokens

chat-pet-sprite-creation

Use when creating or changing VS Code chat pet sprite art, sprite sheets, state animations, eye treatments, Stable/Insiders variants, or pet transitions under src/vs/workbench/contrib/chat/browser/widget/media/chatPet.

microsoft/vscode · 53 tokens

cpu-profile-analysis

Analyze V8/Chrome CPU profiles (.cpuprofile) and DevTools trace files (Trace-.json). Use when: profiling performance, investigating slow functions, comparing code paths, finding bottlenecks, analyzing timeToRequest, understanding call trees from sampling profiler data, analyzing layout/paint/rendering, investigating…

microsoft/vscode · 71 tokens

agent-host-chat-contributions

Build and review cross-cutting agent-host chat behavior through lifecycle contributions. Use when adding turn lifecycle side effects, prompt or context injection, restored-history transformation, protocol-action observation, or when reviewing changes that add code to AgentSideEffects or AgentService.

microsoft/vscode · 56 tokens

auto-perf-optimize

Run agent-driven VS Code performance or memory investigations. Use when asked to launch Code OSS, automate a VS Code scenario, run the Chat memory smoke runner, capture renderer heap snapshots, take workflow screenshots, compare run summaries, or drive a repeatable scenario before heap-snapshot analysis.

microsoft/vscode · 62 tokens