Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add malloydata/publisher --skill eval-diagnosegit clone --depth 1 https://github.com/malloydata/publisherWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/malloydata/publisher/eval-diagnose)<a href="https://agentmods.dev/skills/malloydata/publisher/eval-diagnose"><img src="https://agentmods.dev/badge/skills/malloydata/publisher/eval-diagnose.svg" alt="Measured on agentmods" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00093 | $0.03079 |
| Opus 5 | $0.00046 | $0.01540 |
| Sonnet 5 | $0.00019 | $0.00616 |
| Haiku 4.5 | $0.00009 | $0.00308 |
Grade A, and why
eval-diagnose scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 248 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Diagnose One Answer
Consumes a score event from skill:eval-answer and answers: why did this fail,
and who owns the fix?
Scope boundary: write the diagnosis before any edit exists. This skill never edits a model and never proposes a patch beyond naming the gap. Diagnosis that is allowed to edit becomes justification for an edit somebody already wanted.
Do not diagnose a contaminated attempt or an environment failure. Those are harness or ops, not model work.
Components, in order
Walk in this order and stop at the first with positive evidence. A later
label requires ruling out the earlier ones. Write component with these strings,
never "C1" / "C2" / "C3":
component |
Question |
|---|---|
dataset |
Bad question, bad or missing golden, or environment drift? |
agent-call |
Did the agent ask for the needed concepts, with the right type and scope? |
get_context/model |
Is the needed entity absent, undocumented, weakly labeled, duplicated, or missing guidance? |
get_context/retrieval |
Was an on-target request against a well-described entity ranked or grouped wrong? |
construction |
Did sufficient context arrive, and the agent still built the wrong query? |
model-definition |
Is a measure, join, filter convention, or source semantically wrong? |
owner is separate: model, retrieval, agent-skill, or dataset. There
is no environment owner: an environment failure stops the run before
diagnosis (see the boundary above), so no issue can carry it.
construction requires proving the needed entities and governing guidance were
in the returned context. A server trace proves what Publisher returned, not what
the host kept after compaction. If the rendered tool response is gone, mark
sufficiency unknown and do not assign construction.
Always report construction eligibility as eligible / total. That is a
diagnostic conditional, not a causal comparison.
Step 1: Extract facts from traces, not from memory
For each get_context call, load the stored retrieval trace by the traceId on the
tool_call event. Write down, before you interpret anything:
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago First seen · 248 lines · 93 tokens per session scan A fb39b1400e2c
eval-diagnose is a skill published in the GitHub repository malloydata/publisher (100 stars, last pushed yesterday), licensed MIT. It adds 93 tokens to every session and 3,079 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
dbt-expert
Expert-level dbt (data build tool), models, tests, documentation, incremental models, macros, and Jinja templating. Use when the user mentions analytics engineering, SQL, data transformation, Jinja, or testing, or when the task involves Project Structure and Configuration, Sources and Staging Models, Intermediate and…
reimagine-it-audit
Design Health — runs 19 deterministic quality checks on HTML output. Use when the user says /reimagine-it audit, "audit this page", "check design quality", or wants to verify craft-floor compliance before shipping. Catches blank plates, missing focus rings, non-compositor motion, off-palette accents, dead motion…
reimagine-it-extract
Emit the content signals reimagine-it reads from an HTML file — title, anchors, proper nouns, dates, numbers, emails, links, source hex colors, and the derived palette — as JSON without generating a redesign. Use when the user says /reimagine-it extract, "what does the engine see in this page", "extract the palette"…
reusable-visualization
Build ONE reusable chart visualization component that receives its data and its settings from the host application instead of fetching them, and declares the fields and config options the host exposes to viewers. Use this whenever a single chart component is reused across many different queries rather than built for…
lightdash-agent-slack-messaging
Use this skill when writing, designing, or generating Slack messages for Lightdash's in-app analytics agent. Triggers when someone asks to create agent update messages, Slack digests, agent notifications, weekly summaries, daily summaries, or any Slack copy for the Lightdash project agent. Also use when asked to vary…
developing-in-lightdash
Use when reading, creating, and editing Lightdash dashboards and charts as JSON, including dashboard layout and chart-type-specific configuration.