Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add malloydata/publisher --skill eval-improvegit clone --depth 1 https://github.com/malloydata/publisherWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/malloydata/publisher/eval-improve)<a href="https://agentmods.dev/skills/malloydata/publisher/eval-improve"><img src="https://agentmods.dev/badge/skills/malloydata/publisher/eval-improve/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/malloydata/publisher/eval-improve"><img src="https://agentmods.dev/badge/skills/malloydata/publisher/eval-improve.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00097 | $0.03306 |
| Opus 5.5 | $0.00039 | $0.01322 |
| Sonnet 5.5 | $0.00019 | $0.00661 |
| Haiku 4.5 | $0.00010 | $0.00331 |
Grade A, and why
eval-improve scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 261 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Improve the Model
Takes an issue with owner: model or owner: package-skill and produces one
smallest edit that closes the gap. Every factual claim is backed by a query
you ran.
The two owners license different edits and are never interchangeable:
modelchanges what the data means or how it is described: a.malloyfile.package-skillchanges what the agent is told about using this package: a file under the package'sskills/. Use it for a rule that spans entities or is not a property of any one of them. A fact about a single field belongs in that field's#(doc), which reaches every agent whether or not it loaded a guide.
Both live in the model repo, so one git commit checkpoints and rolls back either.
Two hard boundaries:
- No diagnosis evidence, no edit. If the issue cannot name a concrete gap with a trace or probe, record that and stop. Edits from an empty diagnosis have been the inert and wrong ones.
- This skill never accepts its own edit. You propose and verify. The
acceptance check
in
skill:eval-loopadmits or reverts. An improver writing the query it already knows proves the fix is possible, not that the next blind agent will find it.
Step 0: What the evidence entitles you to change
| Evidence | Edits permitted |
|---|---|
| Verified golden, or a user who states the answer | Any tier. Probes required. Check the golden first. |
| Wrong answer, then a corrected one the user accepted | Prefer docs over structure. The diff between attempts is the missing knowledge. |
| User accepted, later contradicted | Docs, labels, index only. No structural change. |
| Doubt only, or retrieval-only (no verdict) | Docs, labels, index only, and only where the transcript shows a concrete confusion. |
| Silence | No edit. |
A package-skill edit sits at the docs tier throughout: it is permitted
wherever the table permits docs, and never on silence. It is the easiest artifact
in this loop to overfit, because it is prompt text sitting next to the question
it just failed, so the expert test in Step 1 applies to it word for word.
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago Changed · +4 lines 73e98b1fa4c5
- 9d ago Changed · +40 lines · +11 tokens per session 4b8187b7dca2
- 1mo ago Changed · +36 lines 51354bfdd38f
- 1mo ago First seen · 181 lines · 86 tokens per session scan A c6a1caff6009
eval-improve is a skill published in the GitHub repository malloydata/publisher (118 stars, last pushed today), licensed MIT. It adds 97 tokens to every session and 3,306 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
dbt-expert
Expert-level dbt (data build tool), models, tests, documentation, incremental models, macros, and Jinja templating. Use when the user mentions analytics engineering, SQL, data transformation, Jinja, or testing, or when the task involves Project Structure and Configuration, Sources and Staging Models, Intermediate and…
reimagine-it-audit
Design Health — runs 19 deterministic quality checks on HTML output. Use when the user says /reimagine-it audit, "audit this page", "check design quality", or wants to verify craft-floor compliance before shipping. Catches blank plates, missing focus rings, non-compositor motion, off-palette accents, dead motion…
reimagine-it-extract
Emit the content signals reimagine-it reads from an HTML file — title, anchors, proper nouns, dates, numbers, emails, links, source hex colors, and the derived palette — as JSON without generating a redesign. Use when the user says /reimagine-it extract, "what does the engine see in this page", "extract the palette"…
data-validation-contracts
Use when data crosses a boundary into your code (file, API, warehouse table, upload, model input or output), when a join or cleaning step can change row counts, or when adding pandera, Great Expectations, Pydantic or dbt checks.
reusable-visualization
Build ONE reusable chart visualization component that receives its data and its settings from the host application instead of fetching them, and declares the fields and config options the host exposes to viewers. Use this whenever a single chart component is reused across many different queries rather than built for…
repository-writeback
Use when changing the dbt project through a pull request (editDbtProject) or managing those pull requests, offering to fix or extend the semantic layer, or following up a merged pull request.