Borrowing it
Nothing to install: this file belongs to cardstack/boxel. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/cardstack/boxel/main/.claude/skills/skill-change-eval/SKILL.mdgit clone --depth 1 https://github.com/cardstack/boxelWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/cardstack/boxel/skill-change-eval)<a href="https://agentmods.dev/skills/cardstack/boxel/skill-change-eval"><img src="https://agentmods.dev/badge/skills/cardstack/boxel/skill-change-eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/cardstack/boxel/skill-change-eval"><img src="https://agentmods.dev/badge/skills/cardstack/boxel/skill-change-eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00222 | $0.03392 |
| Opus 5.5 | $0.00089 | $0.01357 |
| Sonnet 5.5 | $0.00044 | $0.00678 |
| Haiku 4.5 | $0.00022 | $0.00339 |
Grade A, and why
skill-change-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 233 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Evals for skill changes
The boxel-skills content has two consumers: the in-app AI assistant, which
reads it from the skills realm, and Claude Code, which reads it through the
boxel-skills plugin. They behave differently. Claude Code has a shell,
boxel-cli, and the whole repo. The assistant has run-realm-code, works to
a turn and cost budget, and is graded on both. A rule that helps Claude Code
can make the assistant slower or more expensive, and a test in Claude Code
shows nothing about the assistant.
Every rule has a benefit and a cost. The benefit shows on the builds the rule is for; the cost shows on every build the rule fires on. So a skill change that targets one of the four core use cases ships with evidence of both:
- the problem — an eval fails, without the change, at the thing the change is meant to fix;
- the improvement — the same eval passes that point with the change;
- the cost — the small-card evals (
hello-world,linkedin-profile-computed-edit-restyle) take no more turns, money or time with the change than without it.
A rule whose benefit no eval checks shows only its cost. A Pre-flight rule meant for apps and pages, with no limit in its text, fires on a three-field card too: it adds reads, captures and turns there, and the quality score cannot rise because the small card already scores full marks.
The core use cases:
| # | Use case | What the eval's workspace starts with |
|---|---|---|
| 1 | Create or update card instances | nothing (create), or a definition and instances in initialCards (update) |
| 2 | Create or update card definitions | nothing (create), or the .gts in initialFiles and its instances (update) |
| 3 | Navigate the Boxel UI | the cards the user wants to reach, so the prompt can name them |
| 4 | Bulk update operations | a definition and enough instances that one-at-a-time editing shows (10 or more) |
A change outside these (a typo, a reference page only Claude Code reads, a
pattern nothing loads by default) needs no eval of its own, but it still needs
the cost check if it touches a file the assistant reads on every build:
index.md, the boxel skill, boxel-environment, source-code-editing, the
cardinal rules.
Read the ai-assistant-evals skill and the /run-ai-assistant-eval command
(.claude/commands/run-ai-assistant-eval.md) first. They cover the stack, the
eval users, running, judging, and reading results. This skill covers what to
run and what counts as evidence.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday First seen · 233 lines · 222 tokens per session scan A c02cf0612848
skill-change-eval is a skill published in the GitHub repository cardstack/boxel (27 stars, last pushed today), licensed MIT. It adds 222 tokens to every session and 3,392 once invoked, about $0.0009 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-24.
Other skills, from other repositories
research-engineer
An uncompromising Academic Research Engineer. Operates with absolute scientific rigor, objective criticism, and zero flair. Focuses on theoretical correctness, formal verification, and optimal implementation across any required technology.
tika-eval-compare
Compare extracts from two Tika builds over a corpus to detect regressions in content, encoding, exceptions, and embedded-document handling. Use for "compare before/after extracts", "eval this change against the corpus".
cua-driver
Use Cua Driver for desktop or browser tasks that are awkward or unavailable through Bash/APIs, or when the user explicitly wants GUI interaction: app testing, visual bug reproduction, form filling, calendar entry, screenshots, and demo recording. Also covers Cua setup; not OpenAI Codex Computer Use or web research.
jetson-validate-image
Use after jetson-flash-image to run static BSP checks, on-target smoke/regression tests on a flashed DUT, or both. Not for build or flash steps. Triggers: validate bsp, on-target validation.
atmos-validation
Validate Atmos projects, components, arbitrary JSON Schema inputs, EditorConfig, and GitHub Actions; use affected-file selection and native CI annotations.
skill-benchmark
Benchmark AI skill effectiveness by measuring implementation quality against legacy constraints.