Subwave is an internet radio station where an AI DJ selects music and speaks between tracks for every listener at once. Listeners can request music in plain language, while operators manage the queue, schedule, DJ personas, and station activity. Its catalogue entries define skills and workflows for operating the station.
Borrowing it
Nothing to install: this file belongs to perminder-klair/subwave. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/perminder-klair/subwave/develop/.claude/skills/subwave-llm-bench/SKILL.mdgit clone --depth 1 https://github.com/perminder-klair/subwaveWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/perminder-klair/subwave/subwave-llm-bench)<a href="https://agentmods.dev/skills/perminder-klair/subwave/subwave-llm-bench"><img src="https://agentmods.dev/badge/skills/perminder-klair/subwave/subwave-llm-bench/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/perminder-klair/subwave/subwave-llm-bench"><img src="https://agentmods.dev/badge/skills/perminder-klair/subwave/subwave-llm-bench.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 4 findings, up to high
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- high Privilege Escalation · line 68 Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.Fix: Remove references to credential paths. Use environment variables or secrets managers. For docs, use placeholder paths (e.g., /path/to/config). Never load .env or token files in production code paths.
- high Data Exfiltration · line 70 Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.Fix: Read only explicitly required environment variables by name. Avoid enumerating or copying the full environment, and never log or transmit credentials to untrusted destinations.
- medium MCP Rug Pull · line 150 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
- medium MCP Rug Pull · line 153 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00234 | $0.02426 |
| Opus 5 | $0.00117 | $0.01213 |
| Sonnet 5 | $0.00047 | $0.00485 |
| Haiku 4.5 | $0.00023 | $0.00243 |
Grade C, and why
subwave-llm-bench scanned grade C with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Harvests environment variableshighData exfiltration
Enumerating or grepping the environment for keys collects credentials unrelated to what the mod says it does.
copy it out first: `export OPENROUTER_API_KEY=$(docker exec sub-wave-controller printenv OPENROUTER_API_KEY)`. How it starts
The opening of the file, as written. The whole thing — 186 lines — stays where its author put it; the contents beside it link to each section on GitHub.
SUB/WAVE LLM model benchmark
Two harnesses, one job: measure how well a provider + model handles the station's real LLM calls before trusting it on air.
llm-bench(controller/scripts/llm-bench/,npm run llm-bench) — the primary harness. A matrix runner over every on-air call kind: pool picks (pickNextTrack), agent picks (djAgentPick), pool + agent segments (generateSegment/djAgentSegment), request matching (matchRequest/djAgentRequest), the free-text scripts (intro, link, station ID, hourly), banter, and the programme family (plan, beats, exchanges). Scores reliability + deterministic rule checks, prints a per-model comparison table, writes a diffable JSON report.picker-test.mjs— the legacy picker-only deep-dive. Still useful for high-iteration picker runs comparable with historical results, and it has a bundled orchestration script (see the legacy section at the end).
Both import the live prompts and schemas from src/ (never copies), fake
everything external (library, tools, weather/news data), and override
provider/model only inside their own short-lived process — the live
controller's configured model is never touched.
The two things that surprise people
- Routing matters as much as the model. The same model can pass through
one provider and fail through another, because each
@ai-sdk/*provider translates tools / structured output differently. Canonical case:deepseek-v4-flashscored 0/4 via thedeepseekdirect provider but 4/4 viaopenrouter. Always benchmark through the routing you'll actually deploy — "is model X good?" is the wrong question; "is provider+X good?" is the right one. - The stress scenarios are where truth lives. Easy cells make every model
look fine. The verdict cells are:
djAgentPick/long-context(full prompt + tool loop),generateSegment/dull-weather(the model must decline to air — small models botch the silence encoding; a huge wall-clock here means the structured-output retry rescued a failed first attempt), and the multi-voice JSON kinds (generateBanter,generateProgrammeExchange).
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 9d ago First seen · 186 lines · 234 tokens per session scan C b30338114ac3
subwave-llm-bench is a skill published in the GitHub repository perminder-klair/subwave (1,309 stars, last pushed today), licensed MIT. It adds 234 tokens to every session and 2,426 once invoked, about $0.0012 per session on Opus 5. A static security scan graded it C with 1 finding (harvests environment variables). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
cli-eval
Create and run evaluation suites, watch live benchmark progress, view scorecards, compare model performance, and integrate eval runs with CI workflows from the CLI.
model-merging
Merge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific expertise (math + coding + chat), improving performance beyond single models, or experimenting rapidly with model variants. Covers SLERP, TIES-Merging, DARE, Task…
darwinian-evolver
Evolve prompts/regex/SQL/code with Imbue's evolution loop.
validate
Validate Semantica pipelines, extraction quality, graph schemas, and ontology consistency. Returns structured error/warning checklists. Uses PipelineValidator, PipelineBuilder.validatepipeline(), GraphValidator, and OntologyValidator. Sub-commands: pipeline, step, dependencies, extraction, graph, ontology, performance.
launching-evals
Run, monitor, analyze, and debug LLM evaluations via nemo-evaluator-launcher. Covers running evaluations, checking status and live progress, debugging failed runs, exporting artifacts and logs, and analyzing results. ALWAYS triggers on mentions of running evaluations, checking progress, debugging failed evals…
nemo-automodel-recipe-development
Create and modify NeMo AutoModel training and evaluation recipes, including YAML structure, builders, and execution flow.