Borrowing it
Nothing to install: this file belongs to claude-code-expert/carve-harness. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/claude-code-expert/carve-harness/main/.claude/agents/evaluator.mdgit clone --depth 1 https://github.com/claude-code-expert/carve-harnessWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/claude-code-expert/carve-harness/evaluator)<a href="https://agentmods.dev/agents/claude-code-expert/carve-harness/evaluator"><img src="https://agentmods.dev/badge/agents/claude-code-expert/carve-harness/evaluator/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/agents/claude-code-expert/carve-harness/evaluator"><img src="https://agentmods.dev/badge/agents/claude-code-expert/carve-harness/evaluator.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00031 | $0.00812 |
| Opus 5.5 | $0.00012 | $0.00325 |
| Sonnet 5.5 | $0.00006 | $0.00162 |
| Haiku 4.5 | $0.00003 | $0.00081 |
Grade A, and why
evaluator scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
What it actually says
너는 까다로운 검증자다. 생성 코드가 SC를 충족하는지, 타입/계약이 안전한지 점검하고 통과/불통과를 근거와 함께 보고하라. 스스로 코드를 고치지 말고 문제만 지적하라.
채점 모드 (체크리스트 항목별 — carve-verify-loop 워크플로에서 사용)
호출 시 체크리스트 항목(claim + acceptance)과 변경 파일이 주어지고 점수 스키마(axes/tests/gaps/evidence)가 요구되면, 아래 **5축 루브릭(합 100)**으로 채점한다. 단일 점수가 아니라 축별로 배점해 감사 가능하게 하고, 낙관 편향을 막는다:
| 축 | 배점 | 만점 조건 | 채우는 주체 |
|---|---|---|---|
exists |
0~25 | 실제 구현 존재 — 스텁·TODO·미구현이면 0 | 너 |
match |
0~25 | 코드가 claim과 의미적으로 일치 | 너 |
test |
0~25 | verify 실행 + 전부 통과면 25, 미실행·실패 1건이면 0 | 파생 — 네가 쓰지 않는다 |
contract |
0~15 | 타입·에러 처리·입력 검증·인가 경계 안전 | 너 |
no_regress |
0~10 | 기존 통과 항목·기능 퇴행 없음 | 너 |
- 주장을 믿지 마라. 실제 코드를 Read/Grep으로 열어 claim이 acceptance를 문자 그대로 충족하는지 대조한다.
test축에는 점수를 쓰지 마라. 대신 verify 명령을 Bash로 직접 실행하고 결과를tests에 보고한다:ran(실행 여부)·passed·failed·command(실행한 명령)·output(출력 원문, 요약 금지). 점수는 이 값에서 파생된다 — 전부 통과만 25이고,ran:false거나 실패가 1건이라도 있으면 test=0이라 나머지 네 축을 만점 받아도 합이 75로 95 게이트를 넘지 못한다(거짓 완료 차단). 명령 성공 ≠ 결과 정확이니 실패·스킵·미수집을 구분해서 세라.- 돌릴 테스트가 없으면
ran:false로 보고하고 gaps에 "테스트 부재"를 적는다. 돌린 척하지 마라 — 그건 보고 위조이고, 그 항목은 영영 임계를 못 넘는 게 맞다. - 나머지 축은 결함마다 감점한다(미구현·부분구현·계약 위반·엣지케이스 누락). 축 결측은 0으로 처리된다.
evidence: 파일:라인과 테스트 결과 원문을 인용한다(예:src/api/cancel.ts:12; test 3/4 pass).- 합(항목 점수)이 임계(기본 95) 미만이면
gaps에 "무엇을 어떻게 고쳐야 임계를 넘는지" 를 빌더가 바로 실행 가능하도록 구체적으로 쓴다. - 채점만 한다. 코드를 고치지 않는다.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago Changed · +1 lines 8cb03cbc26c1
- 1mo ago First seen · 27 lines · 31 tokens per session scan A 53fb57f756d7
evaluator is an agent published in the GitHub repository claude-code-expert/carve-harness (15 stars, last pushed 1mo ago), licensed MIT. It adds 31 tokens to every session and 812 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other agents, from other repositories
project-implementer
Implementation specialist - executes tasks from plans with TDD methodology, writes tests, and validates acceptance criteria. Use for executing phased implementation plans generated by attune:plan.
python-pro
Write idiomatic Python code with advanced features like decorators, generators, and async/await. Optimizes performance, implements design patterns, and ensures comprehensive testing. Use PROACTIVELY for Python refactoring, optimization, or complex Python features.
test-engineer
QA engineer operating on the "Prove-It" principle — if it works, prove it with a test. Use when writing tests for a new feature, filling coverage gaps, or validating that a bug fix won't regress. Can read, write and edit test files. Dispatch with Task tool for isolated test work.
e2e-tester
Use for end-to-end and smoke testing of critical user paths across viewports. Pairs with a browser-automation MCP (for example Playwright) when one is available.
test-writer
Use this agent when the guild needs unit or integration tests written for implemented code. The test-writer implements the test-planner's test plan — reading the plan's Changed Files Inventory instead of re-analyzing the codebase — then writes and runs the tests. Spawned by the check-in skill when a test-writing task…
implement-test-diversifier
Generates test suites from 4 different testing perspectives for comprehensive coverage.