calibrate

calibrate is a skill for Claude Code from XRenSiu/claude-code-forge. It costs 359 tokens per session (4,509 once invoked), scanned A, original, MIT.

A skill for checking whether an evaluation standard is trustworthy before using it to approve work. It tests the tests or rubric itself, including whether it works on unseen cases and remains separate from the work being judged.

In plain words
What is it for?
Use it before putting an evaluation-case set or scoring rubric in charge of a release or other gate.
Why use it?
It helps prevent weak or decorative checks from approving incorrect results just because the checks pass.

Skill for Claude Code

Written for Claude Code: user-invocable in frontmatter.

Part of the sdlc plugin — 14 skills, 5 agents shipped together

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/xrensiu/claude-code-forge/calibrate
Any agent
npx skills add XRenSiu/claude-code-forge --skill calibrate
Clone the repo
git clone --depth 1 https://github.com/XRenSiu/claude-code-forge

Made for: Claude Code.

Or install sdlc, the plugin that ships this one along with the rest of its 14 skills, 5 agents.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for calibrate

README.md
[![agentmods](https://agentmods.dev/badge/skills/xrensiu/claude-code-forge/calibrate.svg)](https://agentmods.dev/skills/xrensiu/claude-code-forge/calibrate)
Your own site
<a href="https://agentmods.dev/skills/xrensiu/claude-code-forge/calibrate"><img src="https://agentmods.dev/badge/skills/xrensiu/claude-code-forge/calibrate.svg" alt="Measured on agentmods" height="20"></a>
Per session 359 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 4,509 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00359 $0.04509
Opus 5 $0.00179 $0.02254
Sonnet 5 $0.00072 $0.00902
Haiku 4.5 $0.00036 $0.00451

Measured yesterday against content hash 61de0de30cba, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-06, from the pricing page.

Security

Grade A, and why

calibrate scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

The scan reads SKILL.md. This mod also ships 1 executable file (scripts/verify_calibration.py), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

plugins/sdlc/skills/calibrate/SKILL.md · 237 lines

How it starts

The opening of the file, as written. The whole thing — 237 lines — stays where its author put it; the contents beside it link to each section on GitHub.

calibrate

Prove that a compiled standard — an eval_case set or a rubric_version — is load-bearing, not decorative, before it is allowed to gate anything. This is the meta-gate: the only gate that judges the thing that judges the output. This skill describes the two mirrors that certify a ruler, the two non-negotiables (holdout + isolation) that make the certification real, and the meta-gate verdict that decides whether a standard may go live. It prescribes no step order — the engine sequences the work; what follows are the gaps to fill and the gates that must hold, in any order.

术语映射(在 sdlc 里怎么读这份文件)

本 skill 引自 qanat 仓库,正文保留其领域词汇;在 sdlc 里按下表读:

原文 sdlc 里的对应物
Territory(领地) 一个 bounded context / 模块:dos.yamlbounded_contexts.current_context;issue 的 Depends on DOS 所属上下文
Run(一次执行) 一次 issue → PR 的交付,即 .sdlc/<slug>/ 一个 slug
Contract / Contract 模板 done_when.yaml(v2,以 AC 为单位);模板 = 同类需求复用的 AC 骨架
R001(评估者与执行者隔离) sdlc 的信息隔离:实现子 agent 看不到评审判据;验收在独立会话
R002(闸门资产只能人签) sdlc 的 G2(判据冻结 lock_done_when.py sign --by <人>)与 G3(例外复核)
变更提案 / G2 签字(sdlc) / change proposal / G2 signing (sdlc) / NEEDS_HUMAN assets/change_proposal.md 变更提案 + G2/G3 人签;账本 ledger.md 记 propose
verify_g1 / review_g2(qanat 的机器闸 / 评审闸) sdlc L7 的 A 档机械验收 / C 档判断验收——注意与 sdlc 的 G1(世界裁决)、G2(判据冻结)不是同一对门
MemoryAsset(eval_case / rubric_version / failure_memory) 归档目录 specs/<slug>/ 里的测试集 / 评判 rubric;failure_memory = ledger.md 的 fail 行 + escape-defects.md
daemon / 运行时 本地测试与 CI;/sdlc 的 acceptance 阶段
calibration.resolved 事件 G3 记录里"标准不清"的改判(g3_record.md

The gap (why a green test report is not a calibrated ruler)

A composite of three atoms: Judgment (what counts as a calibrated standard), Capability (run mutation testing / compute an agreement matrix / carve a holdout), and Control (the meta-gate seam — non-compliant forbids activation).

The load-bearing reason this is not free:

Read the full file on GitHub · 237 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 237 lines · 359 tokens per session scan A 61de0de30cba

Subscribe to this mod's changes

calibrate is a skill published in the GitHub repository XRenSiu/claude-code-forge (2 stars, last pushed yesterday), licensed MIT. It adds 359 tokens to every session and 4,509 once invoked, about $0.0018 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-05.

Related

Other skills, from other repositories

research-engineer

An uncompromising Academic Research Engineer. Operates with absolute scientific rigor, objective criticism, and zero flair. Focuses on theoretical correctness, formal verification, and optimal implementation across any required technology.

davila7/claude-code-templates · 43 tokens

tika-eval-compare

Compare extracts from two Tika builds over a corpus to detect regressions in content, encoding, exceptions, and embedded-document handling. Use for "compare before/after extracts", "eval this change against the corpus".

apache/tika · 50 tokens

neuron-evaluation-engineer

Create and run AI evaluations with datasets, assertions, and output drivers in Neuron AI. Use this skill whenever the user mentions evaluation, testing AI systems, creating evaluators, dataset-driven testing, assertion-based validation, or wants to measure AI system performance. Also trigger for tasks involving…

neuron-core/neuron-ai · 77 tokens

jetson-validate-image

Use after jetson-flash-image to run static BSP checks, on-target smoke/regression tests on a flashed DUT, or both. Not for build or flash steps. Triggers: validate bsp, on-target validation.

NVIDIA/skills · 50 tokens

atmos-validation

Validate Atmos projects, components, arbitrary JSON Schema inputs, EditorConfig, and GitHub Actions; use affected-file selection and native CI annotations.

cloudposse/atmos · 31 tokens

skill-benchmark

Benchmark AI skill effectiveness by measuring implementation quality against legacy constraints.

HoangNguyen0403/agent-skills-standard · 16 tokens