Borrowing it
Nothing to install: this file belongs to rushengzhou/sid-code. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/rushengzhou/sid-code/main/.claude/skills/eval-session/SKILL.mdgit clone --depth 1 https://github.com/rushengzhou/sid-codeWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/rushengzhou/sid-code/eval-session)<a href="https://agentmods.dev/skills/rushengzhou/sid-code/eval-session"><img src="https://agentmods.dev/badge/skills/rushengzhou/sid-code/eval-session/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/rushengzhou/sid-code/eval-session"><img src="https://agentmods.dev/badge/skills/rushengzhou/sid-code/eval-session.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00245 | $0.10911 |
| Opus 5.5 | $0.00098 | $0.04364 |
| Sonnet 5.5 | $0.00049 | $0.02182 |
| Haiku 4.5 | $0.00024 | $0.01091 |
Grade A, and why
eval-session scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 306 lines — stays where its author put it; the contents beside it link to each section on GitHub.
eval-session:sid-code 任务会话三段式评估
目的锚点(每次执行前先默念一遍,防止跑偏)
通过一次会话的轨迹,做结果评估 + 过程评估(LLM + harness),找出 sid-code 的 bug 和可优化地方,驱动后续优化。
这句话里有两个常被做丢的重点:
- 是"bug 和可优化地方",不是只找 bug。 任务跑对了但绕了远路、花了冤枉钱、某个 nudge 太弱、缺一个能力——这些都不是缺陷,却正是 sid-code 该优化的地方。只盯"崩没崩"会系统性漏掉一半价值。
- 是"找出 sid-code 的问题",不是"评模型考了几分"。 LLM 过程评估不是目的,是手段——模型的每一次失误/低效,都要回问一句"harness 本可以拦住/让它更容易做对吗?",能翻译成 harness 改进的才是产出。评模型本身,评完就作废了。
术语统一:本 skill 把最终产出统称发现(finding),分两类——缺陷(bug)(行为错误,该修)与优化点(opportunity)(不算错但值得改进:效率/成本/引导强度/能力缺失)。§3 报告两类都要出,别把优化点硬塞进"缺陷"栏,也别丢掉。
核心原则:结果先行,fix 是派生物
评估结果本身就是交付物,不是通往 todo-fix 的过场。 顺序不可颠倒:
三段评估结果(主交付物,要写透)
↓ 人类复核 + 二次确认(gate:未确认不进入下一步)
todo-fix 清单(派生物,每条都必须回指某条已确认的评估结论)
↓ 修复(另开会话,本 skill 不做)
验证修改生效(按 todo-fix 里预置的"验证方法"逐条确认)
三条硬约束:
- 评估结果要写透,不能压成一行表格。 每段用叙述 + 证据展开,像给人看的分析报告,而不是待办清单的附录。宁可结果段长,也不要为了赶 todo-fix 而把判断依据省略。正向样本、边界、"措辞超出证据"这类细节都要保留——它们正是结果可信度的来源。
- todo-fix 的每一条都要可追溯到评估结论。 格式:
fix 项 ← 源自 §X 发现 N。没有对应评估结论的 fix 项不许出现(否则就是凭空臆想的改动)。 - 人类是决策者,skill 只提供依据。 报告必须专门列出"需要人类二次确认的判断项"(见 Phase 6),这些点没拿到人类确认之前,不能当成定论驱动修复。
工具与大模型的职责分层(谁产事实、谁下结论)
评估既依赖脚本也依赖大模型,但分工必须清晰——混了就会要么幻觉数字,要么把工具误报当结论:
| 层 | 谁做 | 产出 | 铁规则 |
|---|---|---|---|
| 确定性层 | trace-digest + 手写 bash/python | 计数、解析、配对、聚合等可复现的事实 + 假设 | 大模型不许口算/臆造数字;该由工具算的必须真跑 |
| 验证层 | 大模型 | 把工具的假设逐条拿去 read 原始数据 + read 源码,证实/证伪 | 工具的假设未经此层验证不许进报告;拒掉假阳性 |
| 判断层 | 大模型 | 严重度、"缺陷 vs 取舍 vs 优化点"边界、修复方向、MEMORY 关联 | 任何脚本都做不了,只能大模型下 |
一句话:工具产"事实+假设",大模型产"结论"。任何结论都不能停在工具未验证的输出上;任何该由工具算的数字都不能由大模型口算。 脚本出错或遗漏时,由验证层兜住——而且脚本自身的缺陷就是一条 harness 发现(如"并行子代理误报循环"),按 §3 写进报告,流程因此自我改进。
目的与边界
对一次已完成的 sid-code 任务会话做系统评估,核心目的是找出 sid-code(harness/工具本身)的 bug 和可优化地方。
本 skill 只评估、只出报告,自己不改任何代码。 针对 sid-code 缺陷/优化点的实施在另一个会话做——本 skill 要为它铺好两件事:① 每个发现预置"验证方法"(修完怎么确认生效);② 明确哪些结论需人类拍板。
区分两个"改代码": "本 skill 不改代码"指的是评估动作本身不动 sid-code 源码。但被评估的那次会话可能已经做了代码改动(如任务含"修复所有缺口")——那些改动是交付物,恰恰是结果评估要严查的对象(见 Phase 2 代码改动五关)。别因为"不改代码"就跳过对被评会话已落地修复的核查。
What ships with it
2 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2mo ago First seen · 306 lines · 245 tokens per session scan A eaadd6ae1096
eval-session is a skill published in the GitHub repository rushengzhou/sid-code (2 stars, last pushed 2mo ago), licensed MIT. It adds 245 tokens to every session and 10,911 once invoked, about $0.0010 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
code-review
Run a thorough self-review pass on the most recent change.
autofix-pr
Watch a pull request and fix it from a worktree of its own (or, with --in-place, the user's own checkout) - CI failures, review feedback from trusted authors and merge conflicts are fixed, pushed to the PR branch and answered on the PR under the user's git and gh identity. Usage - /autofix-pr [ | ] [--once]…
gh-find-prs
Survey open Codewhale PRs and triage each for mergeability and disposition against the real landing branch.
review
Diff-scoped correctness review that reads the codebase around the change — callers, contracts, and invariants — and returns line-anchored findings ranked by severity with confidence, then a merge-risk verdict. Use for reviewing a PR, diff, or named change set. Not for style review, approvals-as-rubber-stamp, or…
simplify
Review recent code changes for reuse, code quality, and efficiency, then directly apply straightforward cleanup improvements. Use when the user wants a post-implementation cleanup pass, pre-PR polish, or asks to simplify/refine recent changes. Invoke with /simplify or /simplify .
cw-slice
Use before writing code for any Codewhale feature, upgrade, or refactor: find the existing owner of the behavior, bound the change to one reviewable slice, and fix the evidence bar before you start.