Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add malloydata/publisher --skill eval-importgit clone --depth 1 https://github.com/malloydata/publisherWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/malloydata/publisher/eval-import)<a href="https://agentmods.dev/skills/malloydata/publisher/eval-import"><img src="https://agentmods.dev/badge/skills/malloydata/publisher/eval-import/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/malloydata/publisher/eval-import"><img src="https://agentmods.dev/badge/skills/malloydata/publisher/eval-import.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00142 | $0.05563 |
| Opus 5.5 | $0.00057 | $0.02225 |
| Sonnet 5.5 | $0.00028 | $0.01113 |
| Haiku 4.5 | $0.00014 | $0.00556 |
Grade A, and why
eval-import scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 24d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 422 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Import questions into an eval set
Questions arrive in whatever shape their author had. This turns them into
evals/<set>/, ready for a run, and records what each key is actually worth.
Scope boundary: cases only. This never answers a question, scores an
answer, or edits a model. It is skill:eval-loop's scrape step, expanded,
and it is where the set's honesty is decided.
The file shapes are defined once, in reference/ledger-schema.md of
skill:eval-answer. Read them there; this skill does not restate them.
reference/case-format.md covers what is specific to an import: which fields
an arriving item maps to, and the four physical formats.
Where the set goes
Two separate rules, and "package" blurs them:
- The same git repository as the model it evaluates, so one commit pins the model and the ledger together. That is what a checkpoint is.
- Never inside the directory tree the answerer's package serves. The answerer can read files in the served tree, and gold there is a contamination path.
So the set is a SIBLING of the package under test, in its repo. A repo with
ecommerce/ under test gets evals/ecommerce-questions/, beside it.
Step 1: keep the file as it arrived
Copy it verbatim into evals/<set>/as-received/, and never edit that copy.
Not the encoding, not the line endings, not an obvious typo.
It is there so a later disagreement is about the key and not about whether
somebody transcribed the question correctly. That disagreement is guaranteed:
skill:eval-loop's reference/auditing-an-answer-key.md is about what happens
when it arrives, and its first check is the question against this file.
If the source cannot be committed, because it carries names, addresses,
internal URLs, or row values somebody pasted into a thread, say so. Record in
set.json where the original lives and a hash of it, note under
sourceNote that the copy is withheld and why, and tell the user that the
byte-for-byte check is now impossible for anyone without the original. Do not
redact silently: a redaction you got wrong becomes the provenance record.
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 24d ago Changed · +179 lines bcd62b5f44ed
- 1mo ago First seen · 243 lines · 142 tokens per session scan A 534c3d383d69
eval-import is a skill published in the GitHub repository malloydata/publisher (118 stars, last pushed today), licensed MIT. It adds 142 tokens to every session and 5,563 once invoked, about $0.0006 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-16.
Other skills, from other repositories
data-validation-contracts
Use when data crosses a boundary into your code (file, API, warehouse table, upload, model input or output), when a join or cleaning step can change row counts, or when adding pandera, Great Expectations, Pydantic or dbt checks.
mcp-artifact-integration
Integrate Lightdash MCP query results into Claude-created HTML or React artifacts. Covers host bridges, response envelopes, polling, errors, CSV fallback and chart data. Not needed for ordinary chat or Lightdash's built-in chart app.
effective-dbt-sql
Use when writing or modifying dbt model SQL — deciding whether to add a subquery or reuse an existing dimension/metric, structuring a query, joining models, or refactoring a metric's SQL. Encodes SQL semantic-correctness rules: reuse existing fields, prefer CTEs over correlated subqueries, and make joins and column…
dbt-expert
Expert-level dbt (data build tool), models, tests, documentation, incremental models, macros, and Jinja templating. Use when the user mentions analytics engineering, SQL, data transformation, Jinja, or testing, or when the task involves Project Structure and Configuration, Sources and Staging Models, Intermediate and…
alibabacloud-data-agent-mcp-skill
Alibaba Cloud Data Agent MCP skill (alibabacloud-data-agent-mcp-skill, data-agent MCP) for enterprise database/file analysis. Use when the user asks (in any language, including Chinese) to query/analyze DMS-managed databases, run SQL/data analysis, start quick-query (lite) or deep-analysis (pro/ultra) sessions…
wren-usage
Wren Engine — semantic SQL engine for AI agents. Query 22+ data sources (PostgreSQL, BigQuery, Snowflake, MySQL, ClickHouse, etc.) through a modeling layer (MDL). This skill is the main entry point: it guides setup, delegates to focused sub-skills for SQL authoring, MDL generation, project management, and MCP server…