Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/malloydata/publisher/eval-improvenpx skills add malloydata/publisher --skill eval-improvegit clone --depth 1 https://github.com/malloydata/publisherWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/malloydata/publisher/eval-improve)<a href="https://agentmods.dev/skills/malloydata/publisher/eval-improve"><img src="https://agentmods.dev/badge/skills/malloydata/publisher/eval-improve.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00086 | $0.02149 |
| Opus 5 | $0.00043 | $0.01074 |
| Sonnet 5 | $0.00017 | $0.00430 |
| Haiku 4.5 | $0.00009 | $0.00215 |
Grade A, and why
eval-improve scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 181 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Improve the Model
Takes an issue with owner: model and produces one smallest edit that closes
the gap. Every factual claim is backed by a query you ran.
Two hard boundaries:
- No diagnosis evidence, no edit. If the issue cannot name a concrete gap with a trace or probe, record that and stop. Edits from an empty diagnosis have been the inert and wrong ones.
- This skill never accepts its own edit. You propose and verify. The
acceptance check
in
skill:eval-loopadmits or reverts. An improver writing the query it already knows proves the fix is possible, not that the next blind agent will find it.
Step 0: What the evidence entitles you to change
| Evidence | Edits permitted |
|---|---|
| Verified golden, or a user who states the answer | Any tier. Probes required. Check the golden first. |
| Wrong answer, then a corrected one the user accepted | Prefer docs over structure. The diff between attempts is the missing knowledge. |
| User accepted, later contradicted | Docs, labels, index only. No structural change. |
| Doubt only, or retrieval-only (no verdict) | Docs, labels, index only, and only where the transcript shows a concrete confusion. |
| Silence | No edit. |
Do not edit for BAD-REFERENCE or AMBIGUOUS-REFERENCE. Those are the
golden side door in skill:eval-loop: repair or hold the golden, bump
goldenRevision on the case, and open a new baseline run. Being right and unmatched
beats encoding a defect or an unsettled key. Do not edit for a skill,
retrieval, or dataset owner.
Step 1: What a correct answer may teach
Encode what a domain expert would volunteer unprompted: systems of record, vocabulary to stored codes, what a metric means and at what grain, which relationship is the real one.
The expert test, per edit: would a domain expert have said this about their data with no question in front of them? Reject:
- a field that hard-codes this question's filter and serves no other question
- this question's text, qid, or expected numbers in a doc, comment, or name
- a join copied from gold SQL that you have not probed as a real relationship
What ships with it
2 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday First seen · 181 lines · 86 tokens per session scan A c6a1caff6009
eval-improve is a skill published in the GitHub repository malloydata/publisher (99 stars, last pushed yesterday), licensed MIT. It adds 86 tokens to every session and 2,149 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
reusable-visualization
Build ONE reusable chart visualization component that receives its data and its settings from the host application instead of fetching them, and declares the fields and config options the host exposes to viewers. Use this whenever a single chart component is reused across many different queries rather than built for…
debug-local
Debug the Lightdash app using PM2 logs, Maple traces, and browser automation. Use when investigating issues, tracking down bugs, understanding request flow, or correlating frontend actions with backend behavior.
frontend-style-guide
Apply the Lightdash frontend style guide and design principles when working on React components or styling frontend code. Use when editing TSX files, building or reviewing UI, fixing styling issues, or when the user mentions Mantine, design, styling, or CSS modules.
ld-permissions
Guide for Lightdash's CASL-based authorization system. Use when working with scopes, custom roles, abilities, permissions, ForbiddenError, authorization, or access control. Helps with adding new scopes, debugging permission issues, understanding the permission flow, and creating custom roles.
lightdash-agent-slack-messaging
Use this skill when writing, designing, or generating Slack messages for Lightdash's in-app analytics agent. Triggers when someone asks to create agent update messages, Slack digests, agent notifications, weekly summaries, daily summaries, or any Slack copy for the Lightdash project agent. Also use when asked to vary…
developing-in-lightdash
Use when reading, creating, and editing Lightdash dashboards and charts as JSON, including dashboard layout and chart-type-specific configuration.