eval

A command for defining and checking evaluations: repeatable checks that measure whether a feature works and whether existing behaviour still works.

In plain words
What is it for?
Use it to define evaluations for a feature, run capability and regression checks, list them, and generate a progress report.
Why use it?
It gives feature work explicit success criteria and records passed and failed checks instead of relying only on informal testing.

Command

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add commands/codelably/harmony-claude-code/eval
Clone the repo
git clone --depth 1 https://github.com/codelably/harmony-claude-code
Per session 0 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 800 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00000 $0.00800
Opus 5 $0.00000 $0.00400
Sonnet 5 $0.00000 $0.00160
Haiku 4.5 $0.00000 $0.00080

Measured 2d ago against content hash 37e2158fc3fe, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

commands/eval.md · 121 lines

What it actually says

评测命令(Eval Command)

管理评测驱动开发(eval-driven development)工作流。

用法(Usage)

/eval [define|check|report|list] [feature-name]

定义评测(Define Evals)

/eval define feature-name

创建一个新的评测定义:

  1. 创建 .claude/evals/feature-name.md 文件,使用以下模板:
## EVAL: feature-name
创建时间:$(date)

### 能力评测(Capability Evals)
- [ ] [能力描述 1]
- [ ] [能力描述 2]

### 回归评测(Regression Evals)
- [ ] [现有行为 1 仍然正常工作]
- [ ] [现有行为 2 仍然正常工作]

### 通过准则(Success Criteria)
- 能力评测(capability evals)的 pass@3 > 90%
- 回归评测(regression evals)的 pass^3 = 100%
  1. 提示用户填写具体准则。

检查评测(Check Evals)

/eval check feature-name

运行特定功能的评测:

  1. .claude/evals/feature-name.md 读取评测定义。
  2. 对于每一项能力评测:
    • 尝试验证准则。
    • 记录 PASS/FAIL。
    • .claude/evals/feature-name.log 中记录尝试日志。
  3. 对于每一项回归评测:
    • 运行相关测试。
    • 与基准(baseline)进行对比。
    • 记录 PASS/FAIL。
  4. 报告当前状态:
EVAL CHECK: feature-name
========================
能力(Capability): X/Y 通过
回归(Regression): X/Y 通过
状态(Status): 进行中(IN PROGRESS)/ 已就绪(READY)

生成报告(Report Evals)

/eval report feature-name

生成完整的评测报告:

EVAL REPORT: feature-name
=========================
生成时间:$(date)

能力评测(CAPABILITY EVALS)
----------------
[eval-1]: PASS (pass@1)
[eval-2]: PASS (pass@2) - 需重试
[eval-3]: FAIL - 见备注

回归评测(REGRESSION EVALS)
----------------
[test-1]: PASS
[test-2]: PASS
[test-3]: PASS

指标(METRICS)
-------
能力 pass@1: 67%
能力 pass@3: 100%
回归 pass^3: 100%

备注(NOTES)
-----
[任何问题、边界情况或观察结果]

建议(RECOMMENDATION)
--------------
[可发布(SHIP)/ 需改进(NEEDS WORK)/ 阻塞(BLOCKED)]

列出评测(List Evals)

/eval list

显示所有评测定义:

EVAL DEFINITIONS
================
feature-auth      [3/5 通过] 进行中(IN PROGRESS)
feature-search    [5/5 通过] 已就绪(READY)
feature-export    [0/4 通过] 未开始(NOT STARTED)

参数(Arguments)

$ARGUMENTS:

  • define <name> - 创建新的评测定义。
  • check <name> - 运行并检查评测。
  • report <name> - 生成完整报告。
  • list - 显示所有评测。
  • clean - 清除旧的评测日志(保留最近 10 次运行记录)。
Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 121 lines · 0 tokens per session scan A 37e2158fc3fe

Subscribe to this mod's changes

eval is a command published in the GitHub repository codelably/harmony-claude-code (42 stars, last pushed 6mo ago), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 800 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.