Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/uipath/coder_eval/initnpx skills add UiPath/coder_eval --skill initgit clone --depth 1 https://github.com/UiPath/coder_evalWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00043 | $0.01549 |
| Opus 5 | $0.00022 | $0.00775 |
| Sonnet 5 | $0.00009 | $0.00310 |
| Haiku 4.5 | $0.00004 | $0.00155 |
Grade A, and why
init scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 133 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Set up coder-eval in this repository
Goal: leave the user with a task directory containing one real task they can run immediately, not an empty scaffold. The task must exercise something this repository actually ships.
The user's request is: $ARGUMENTS
Step 1 — Check prerequisites
Run coder-eval --version. Installing this plugin did not install the CLI, and
every later step needs it.
If it is missing, follow ${CLAUDE_PLUGIN_ROOT}/reference/cli-setup.md: offer the
install, ask before running it, and confirm with coder-eval --version
afterwards. Never install unprompted, and do not continue if the user declines.
That reference also covers the other half of the version check — whether this project pins a coder-eval version, and what to do when the installed one does not match it.
Step 2 — Check whether this repository is already configured
This skill scaffolds a first suite. Run against a repository that already has one, it would write a "first task" beside an existing tree and report success — so find out before writing anything.
Locate the eval tree per ${CLAUDE_PLUGIN_ROOT}/reference/repo-layout.md. If the
repository already has tasks, report the inventory and stop: how many task files and
where, how many run directories, and whether there are experiments. Then point the user
at the skills that act on what exists —
/coder-eval:lint-tasksto review the tasks already there;/coder-eval:analyzeto read a finished run.
Only scaffold if the repository has none, or if the user explicitly asks for more after seeing the inventory. Never overwrite an existing task file, whatever they ask for; add alongside it.
A repository with tasks but no experiments is partially configured, not empty. Report that as it is and offer the missing piece — do not scaffold a first task as though there were nothing there.
Step 3 — Scan for what is testable
Look for these, in priority order, and report what you found before writing anything:
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 133 lines · 43 tokens per session scan A 087ec6901986
init is a skill published in the GitHub repository UiPath/coder_eval (119 stars, last pushed 4d ago), licensed Apache-2.0. It adds 43 tokens to every session and 1,549 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
execution
M-1.4 execution skill — 跑 single task 产 patch + 提交 envelope。.
fix-self-check
M-1.6 envelope self-check——独立性保证不自欺欺人 (5 blockingcheck)。由 CLI ./tw fix complete --self-check-mode fork(默认即 fork)自动派起,不经 Skill 工具调用;fix 主会话产 FixCompleted 前直读本文,是为理解双层验证关系。.
review
M-1.5 review skill — 在 patch 跟 contract 之间找 finding，produce Finding 一等对象。.
dependency-analyze
从 task 的 read/write set + concept statemachine 推导 6 种依赖类型的提案。主 planner 决定边的真实性。派它时只给 read/write set 与疑点、不给预期边集；已有预判逐条标「待复核」交它取证。.
execution-self-check
Pre-submit 自检——envelope 提交 commit gate 前必跑。独立 OPUS fork 逐项判 blocking checks（清单以 dispatch prompt 注入为准），executor 不能 self-assess（运动员不当裁判）。.
fix
修复者 — 把一条被发现的问题（finding）按它的闭合合约修干净，修一个不制造下一个。产 FixProposed + 临时的 FixCompleted，不自判问题关闭（那是复查的权）。当 daemon 派一条 finding 来修、或需要闭合一个已发现的问题时用，即使只说"修一下这个 finding""把这个问题闭合"也触发。调用名就是 fix（Skill 工具）或 /fix（命令），没有 harness: 之类的前缀。.