Getting it into your agent
It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.
git clone --depth 1 https://github.com/malloydata/publishernpx agentmods add skills/malloydata/publisher/eval-reportWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/malloydata/publisher/eval-report)<a href="https://agentmods.dev/skills/malloydata/publisher/eval-report"><img src="https://agentmods.dev/badge/skills/malloydata/publisher/eval-report/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/malloydata/publisher/eval-report"><img src="https://agentmods.dev/badge/skills/malloydata/publisher/eval-report.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00103 | $0.06227 |
| Opus 5.5 | $0.00041 | $0.02491 |
| Sonnet 5.5 | $0.00021 | $0.01245 |
| Haiku 4.5 | $0.00010 | $0.00623 |
Grade A, and why
eval-report scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 14d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
If the truth server is not running it says so and prints the `curl` to run How it starts
The opening of the file, as written. The whole thing — 459 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Report a Run
A run directory is JSONL and a console summary that scrolls away. Neither is a result. This skill turns one into something a person can open, and it is the last step of every run, including a run that failed.
The complaint this exists to answer, in a reviewer's words: "eval runs and a bunch of stuff happens and it is hard to know the actual results."
Scope boundary: this skill reports what the ledger already says. It does not
score an answer (skill:eval-answer), decide who owns a failure
(skill:eval-diagnose), or edit a model (skill:eval-improve). If a number is
not in the ledger, do not put it in the report.
The report is about THIS RUN, not about the harness. Defects you find in the eval tooling while running it are real and worth filing, and they do not belong here: the reader wants to know what their model scored and why, not what is wrong with the thing that measured it. File those against the harness. The one exception is anything that qualifies THIS run's number -- a truncated attempt, a contaminated one, an unestablished answer key -- which the "Eval failures" section exists for.
Do not audit the harness. You were asked to measure a model. This is the most common way this job goes wrong, and it does not look like going wrong: a run turns up something odd in a script, the odd thing is genuinely a bug, and the reply comes back as a critique of the tooling with the model's score somewhere underneath. The reader asked what their model scored. Answer that.
So, unless the user asked you to work on the harness:
- Do not read harness source to satisfy your own curiosity about a number. Read it when a number you must report cannot be explained any other way, and stop when it can.
- Do not propose harness fixes, refactors, flags or "while I was in there" improvements. Not in the report, not in the chat reply.
- When a harness defect DID change this run's number, the report gets one sentence: what the number should be and why. Not the mechanism, not the file, not the fix.
- Keep a defect that changed nothing out of the report entirely. Mention it once in chat, in a line, and let the user decide whether they want it chased.
A harness bug you found and did not chase is not a loose end. It is the job being done. If the user wants it fixed they will say so, and then it is a different task with its own turn.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 14d ago Changed · +8 lines 5c81778c6650
- 15d ago Changed 07f46c712556
- 21d ago Changed ce025c6aff75
- 22d ago Changed · +89 lines a22a1287d564
- 24d ago First seen · 362 lines · 103 tokens per session scan A 9518f2566a41
eval-report is a skill published in the GitHub repository malloydata/publisher (118 stars, last pushed today), licensed MIT. It adds 103 tokens to every session and 6,227 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-22.
Other skills, from other repositories
reimagine-it-extract
Emit the content signals reimagine-it reads from an HTML file — title, anchors, proper nouns, dates, numbers, emails, links, source hex colors, and the derived palette — as JSON without generating a redesign. Use when the user says /reimagine-it extract, "what does the engine see in this page", "extract the palette"…
reusable-visualization
Build ONE reusable chart visualization component that receives its data and its settings from the host application instead of fetching them, and declares the fields and config options the host exposes to viewers. Use this whenever a single chart component is reused across many different queries rather than built for…
developing-in-lightdash
Use when reading, creating, and editing Lightdash dashboards and charts as JSON, including dashboard layout and chart-type-specific configuration.
sdk-features
Map Lightdash SDK capabilities to their host-UI names and the app-code wiring each needs. Use when the user asks about a feature by name (Inspect data, drill-down, exports, shareable URLs), when offering newly available features after a template upgrade, or when wiring a host-facing capability into the app.
upgrade-preflight
Checks whether a self-hosted Lightdash upgrade is safe to run, and reads the tooling's answer without over-reading it. Use when upgrading a self-hosted instance, planning a maintenance window, answering "is this upgrade safe", or recovering a failed, hung, parked or lock-stuck migration — covers lightdash…
developing-data-apps-locally
Use when editing a locally created or downloaded Lightdash data app — how local editing, building, and uploading work, and what is read-only.