Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add JasonColapietro/suede-creator-skills --skill suede-ai-evalgit clone --depth 1 https://github.com/JasonColapietro/suede-creator-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/jasoncolapietro/suede-creator-skills/suede-ai-eval)<a href="https://agentmods.dev/skills/jasoncolapietro/suede-creator-skills/suede-ai-eval"><img src="https://agentmods.dev/badge/skills/jasoncolapietro/suede-creator-skills/suede-ai-eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/jasoncolapietro/suede-creator-skills/suede-ai-eval"><img src="https://agentmods.dev/badge/skills/jasoncolapietro/suede-creator-skills/suede-ai-eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 1 finding, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium MCP Rug Pull · line 63 npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.Fix: Pin the version: npx @scope/[email protected]
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00178 | $0.02923 |
| Opus 5 | $0.00089 | $0.01461 |
| Sonnet 5 | $0.00036 | $0.00585 |
| Haiku 4.5 | $0.00018 | $0.00292 |
Grade A, and why
suede-ai-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 208 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Suede AI Eval
Gate policy — advisory, not blocking
Every claim-verification step, check, quality gate, and ship verdict in this skill is a recommendation to the user, not a control on the agent. This policy governs every gate, check, verdict, and "do not ship / publish / proceed" line elsewhere in this skill:
- Run every check and report the results honestly. Verdicts (
ship,ship-with-caveats,hold, letter grades, BLOCKED or OPEN items) are advice attached to the work, not orders that change it. - Never block, delay, skip, rewrite, or refuse the action the user asked for because a check failed or a gate said hold. Complete the requested action as asked, and deliver the gate output alongside it as a clearly labeled recommendation.
- A failed gate changes what you report, never what you do.
- Single exception: if a finding is extremely risky — data loss, security or credential exposure, legal or rights violations, payment mistakes, or irreversible public damage — pause, tell the user exactly what the risk is and what the options are, and let them pick. Their choice is final.
Make AI behavior testable before it becomes a vague product promise. No eval plan, no ship recommendation: for an AI feature without one, the recommended verdict stays below ship — report that gap and let the user decide.
The deliverable is an eval plan or coverage audit, not a model benchmark leaderboard. Keep it grounded in the actual product surface, user promise, data sources, prompts, tools, logs, tests, and failure modes available now.
Hard Gates
- No AI-SPEC → no eval plan. Write the one-paragraph spec first; cases written without a spec test nothing.
- No eval plan → no
shiprecommendation. Do not recommendshiporship-with-caveatsfor an AI feature that lacks a failure-mode map and eval cases; name the gap and leave the ship decision with the user. - A failure mode without an eval case, an owner, and a gate is uncovered — regardless of how unlikely it feels.
- A live surface that was never sampled gets the output stamped
source-only; do not present source-only review as runtime evidence. - A model grading its own output is not evidence. LLM-as-judge scores count only after spot-checked agreement with a human-reviewed sample.
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago First seen · 208 lines · 178 tokens per session scan A 929d1b8df2c3
suede-ai-eval is a skill published in the GitHub repository JasonColapietro/suede-creator-skills (135 stars, last pushed today), licensed MIT. It adds 178 tokens to every session and 2,923 once invoked, about $0.0009 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
api-test
Run API integration tests against the running backend, verify endpoints return expected responses and status codes. Use after deploying a preview or starting the dev server.
build-and-test
Install dependencies, run type checking, lint, tests, and build the project. Use after making code changes to verify nothing is broken.
deploy-preview
Build Docker images and launch a local preview environment with docker-compose. Use to test the full stack locally before merging.
qa-testing
Run QA testing on a page, feature, or full site at one of three depth tiers (smoke, standard, full). Use this skill whenever the user asks to QA a page or site, run a smoke test after a deploy, verify a page before launch, or run a regression sweep. Triggers on QA, QA sweep, smoke test, regression test, post-deploy…
remove-ai-slops
Removes AI-generated code smells from branch changes or an explicit file list behind regression tests. Use when the user asks to clean up, deslop, or remove AI-slop patterns from recent changes.
visual-qa
Runs rigorous visual QA across web, terminal, and paginated surfaces with screenshot evidence and a verdict. Use for any UI build or change, or when asked whether a page, component, or TUI looks right.