Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/codelably/harmony-claude-code/eval-harnessnpx skills add codelably/harmony-claude-code --skill eval-harnessgit clone --depth 1 https://github.com/codelably/harmony-claude-codeWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00019 | $0.01718 |
| Opus 5 | $0.00010 | $0.00859 |
| Sonnet 5 | $0.00004 | $0.00344 |
| Haiku 4.5 | $0.00002 | $0.00172 |
Grade A, and why
eval-harness scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 228 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval Harness 技能
Claude Code 工作階段的正式評估框架，實作 eval 驅動開發（EDD）原則。
理念
Eval 驅動開發將 evals 視為「AI 開發的單元測試」：
- 在實作前定義預期行為
- 開發期間持續執行 evals
- 每次變更追蹤回歸
- 使用 pass@k 指標進行可靠性測量
Eval 類型
能力 Evals
測試 Claude 是否能做到以前做不到的事：
[CAPABILITY EVAL: feature-name]
任務：Claude 應完成什麼的描述
成功標準：
- [ ] 標準 1
- [ ] 標準 2
- [ ] 標準 3
預期輸出：預期結果描述
回歸 Evals
確保變更不會破壞現有功能：
[REGRESSION EVAL: feature-name]
基準：SHA 或檢查點名稱
測試：
- existing-test-1: PASS/FAIL
- existing-test-2: PASS/FAIL
- existing-test-3: PASS/FAIL
結果：X/Y 通過（先前為 Y/Y）
評分器類型
1. 基於程式碼的評分器
使用程式碼的確定性檢查：
# 檢查檔案是否包含預期模式
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
# 檢查測試是否通過
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
# 檢查建置是否成功
npm run build && echo "PASS" || echo "FAIL"
2. 基於模型的評分器
使用 Claude 評估開放式輸出：
[MODEL GRADER PROMPT]
評估以下程式碼變更：
1. 它是否解決了陳述的問題？
2. 結構是否良好？
3. 邊界案例是否被處理？
4. 錯誤處理是否適當？
分數：1-5（1=差，5=優秀）
理由：[解釋]
3. 人工評分器
標記為手動審查：
[HUMAN REVIEW REQUIRED]
變更：變更內容的描述
理由：為何需要人工審查
風險等級：LOW/MEDIUM/HIGH
指標
pass@k
「k 次嘗試中至少一次成功」
- pass@1：第一次嘗試成功率
- pass@3：3 次嘗試內成功
- 典型目標：pass@3 > 90%
pass^k
「所有 k 次試驗都成功」
- 更高的可靠性標準
- pass^3：連續 3 次成功
- 用於關鍵路徑
Eval 工作流程
1. 定義（編碼前）
## EVAL 定義：feature-xyz
### 能力 Evals
1. 可以建立新使用者帳戶
2. 可以驗證電子郵件格式
3. 可以安全地雜湊密碼
### 回歸 Evals
1. 現有登入仍可運作
2. 工作階段管理未變更
3. 登出流程完整
### 成功指標
- 能力 evals 的 pass@3 > 90%
- 回歸 evals 的 pass^3 = 100%
2. 實作
撰寫程式碼以通過定義的 evals。
3. 評估
# 執行能力 evals
[執行每個能力 eval，記錄 PASS/FAIL]
# 執行回歸 evals
npm test -- --testPathPattern="existing"
# 產生報告
4. 報告
EVAL 報告：feature-xyz
========================
能力 Evals：
create-user: PASS (pass@1)
validate-email: PASS (pass@2)
hash-password: PASS (pass@1)
整體： 3/3 通過
回歸 Evals：
login-flow: PASS
session-mgmt: PASS
logout-flow: PASS
整體： 3/3 通過
指標：
pass@1: 67% (2/3)
pass@3: 100% (3/3)
狀態：準備審查
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 228 lines · 19 tokens per session scan A 4333f5558ca9
eval-harness is a skill published in the GitHub repository codelably/harmony-claude-code (42 stars, last pushed 6mo ago), licensed MIT. It adds 19 tokens to every session and 1,718 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
systematic-debugging
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes.
brainstorming
You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. Explores user intent, requirements and design before implementation.
chat-pet-sprite-creation
Use when creating or changing VS Code chat pet sprite art, sprite sheets, state animations, eye treatments, Stable/Insiders variants, or pet transitions under src/vs/workbench/contrib/chat/browser/widget/media/chatPet.
cpu-profile-analysis
Analyze V8/Chrome CPU profiles (.cpuprofile) and DevTools trace files (Trace-.json). Use when: profiling performance, investigating slow functions, comparing code paths, finding bottlenecks, analyzing timeToRequest, understanding call trees from sampling profiler data, analyzing layout/paint/rendering, investigating…
agent-host-chat-contributions
Build and review cross-cutting agent-host chat behavior through lifecycle contributions. Use when adding turn lifecycle side effects, prompt or context injection, restored-history transformation, protocol-action observation, or when reviewing changes that add code to AgentSideEffects or AgentService.
auto-perf-optimize
Run agent-driven VS Code performance or memory investigations. Use when asked to launch Code OSS, automate a VS Code scenario, run the Chat memory smoke runner, capture renderer heap snapshots, take workflow screenshots, compare run summaries, or drive a repeatable scenario before heap-snapshot analysis.