Borrowing it
Nothing to install: this file belongs to cloudrift-ai/emmy. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/cloudrift-ai/emmy/main/.agents/skills/reproduce-article-benchmarks/SKILL.mdgit clone --depth 1 https://github.com/cloudrift-ai/emmyWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/cloudrift-ai/emmy/reproduce-article-benchmarks)<a href="https://agentmods.dev/skills/cloudrift-ai/emmy/reproduce-article-benchmarks"><img src="https://agentmods.dev/badge/skills/cloudrift-ai/emmy/reproduce-article-benchmarks/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/cloudrift-ai/emmy/reproduce-article-benchmarks"><img src="https://agentmods.dev/badge/skills/cloudrift-ai/emmy/reproduce-article-benchmarks.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 2 findings, up to high
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- high Supply Chain · line 109 Remote code is downloaded and executed. This bypasses code review and could introduce malicious code.Fix: Avoid downloading and executing remote scripts. Use trusted packages from PyPI/npm. If remote fetch is required, verify checksums and use HTTPS.
- medium Rogue Agent · line 209 Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.Fix: Remove any persistence mechanisms (cron jobs, startup scripts, state files). Skills should not maintain state across sessions without explicit user consent.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00095 | $0.04640 |
| Opus 5 | $0.00048 | $0.02320 |
| Sonnet 5 | $0.00019 | $0.00928 |
| Haiku 4.5 | $0.00010 | $0.00464 |
Grade A, and why
reproduce-article-benchmarks scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
curl -s "https://hub.docker.com/v2/repositories/<org>/<repo>/tags?page_size=25&ordering=last_updated" \ How it starts
The opening of the file, as written. The whole thing — 301 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Reproduce a published article's benchmarks
A published article's benchmarks already live in this repo as experiments. This skill re-runs them and diffs the result against the article — it does not re-derive the benchmarks and it does not invent new ones.
Read the article at run time. Never work from a transcribed copy of its numbers. No baseline files, no cached tables, no numbers memorized into this skill. A snapshot of an article diverges from it silently, and a stale reference number invents regressions that were never there. The article is the reference; fetch it every run.
The deliverable is one comparison: measured vs published, per cell, with the setup that produced it stated plainly. A regression is a finding to hand back, never something to diagnose or fix inside the run — a repro session's entire value is that it is a clean, uncontaminated measurement.
Step 0 — A scratch directory
Later steps write logs and modified recipe copies to $SCRATCH. Set it first: unset, "$SCRATCH/<suite>" expands
to /<suite>, so a recipe override lands at the filesystem root instead of somewhere disposable. Use the session's
scratchpad directory if the environment provides one, otherwise make one:
export SCRATCH="${SCRATCH:-$(mktemp -d -t emmy-repro)}"; echo "$SCRATCH"
Keep results out of the checkout — a repro run should leave the working tree exactly as it found it.
Step 1 — Read the article
Fetch the article URL and extract, explicitly:
- every results table, with its exact numbers, units, and column meaning (tok/s vs ms; mean vs median vs p99);
- the lanes each table compares (stock engine / emmy / a precision fork / a third-party engine / a speculation depth) — these become the run's control lanes;
- the workload points (input/output lengths, concurrency, batch sizes, kernel shapes) and the client settings;
- the hardware and stack it was measured on (GPU, driver, CUDA, torch, engine version);
- any methodology caveat the article states about itself — "compare rows of this table only with each other", "this grid ran on a different machine", "this cell failed and here is why". These are load-bearing; a caveat ignored turns a correct result into a false regression.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago Changed 40f719de7b5c
- 11d ago First seen · 301 lines · 95 tokens per session scan A e7714faff92e
reproduce-article-benchmarks is a skill published in the GitHub repository cloudrift-ai/emmy (80 stars, last pushed today), licensed Apache-2.0. It adds 95 tokens to every session and 4,640 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
research-engineer
An uncompromising Academic Research Engineer. Operates with absolute scientific rigor, objective criticism, and zero flair. Focuses on theoretical correctness, formal verification, and optimal implementation across any required technology.
promptfoo-redteam-setup
Create or refine promptfoo redteam setup configs: purpose, targets, plugins, strategies, frameworks, multi-input target inputs, policy text, grader guidance, contexts, and static-code-derived target/threat mapping. Use when preparing a red team scan plan from live probes, code evidence, or provider configs, or when…
train-pose
Train/evaluate WiFi pose models honestly — camera-supervised (MediaPipe + CSI) and camera-free (WiFlow), always checked against the mean-pose baseline before any PCK is quoted.
promptfoo-provider-setup
Configure promptfoo providers or redteam targets for hosted models, live HTTP APIs, Python/JavaScript local scripts, agent SDKs, or multi-input systems. Use when connecting promptfoo to the system under test, mapping vars, auth env vars, request bodies, response transforms, or static-code-derived provider wrappers. Do…
promptfoo-redteam-run
Run, rerun, inspect, and QA promptfoo redteam scans from generated redteam YAML or an existing redteam setup config. Use when executing promptfoo redteam eval or promptfoo redteam run, exporting results, triaging attack success rate, grader failures, target errors, filter/rerun commands, reports, or CI gates. Do not…
experiment-audit
A review step that checks whether an experiment's reported results are supported by real data and a sufficiently broad test.