Tons of Skills is a model-agnostic marketplace that distributes reusable skills, plugins, agents, commands, hooks, and settings for coding-agent tools. It is intended for people who want to browse, install, and manage agent extensions, with Claude Code as its verified native harness. The catalogue entries are extensions provided by or associated with this marketplace.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplaceWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/jeremylongshore/tons-of-skills-marketplace/evals)<a href="https://agentmods.dev/agents/jeremylongshore/tons-of-skills-marketplace/evals"><img src="https://agentmods.dev/badge/agents/jeremylongshore/tons-of-skills-marketplace/evals/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/agents/jeremylongshore/tons-of-skills-marketplace/evals"><img src="https://agentmods.dev/badge/agents/jeremylongshore/tons-of-skills-marketplace/evals.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00059 | $0.00779 |
| Opus 5.5 | $0.00024 | $0.00312 |
| Sonnet 5.5 | $0.00012 | $0.00156 |
| Haiku 4.5 | $0.00006 | $0.00078 |
Grade A, and why
evals scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 77 lines — stays where its author put it; the contents beside it link to each section on GitHub.
You are Evals — LLM Evaluation Engineer on the AI Operations Team. Eval harness design, benchmark suites, automated regression, human eval pipelines.
Think in production reliability, cost efficiency, and measurable quality. Every AI system recommendation must be paired with an eval or metric that proves it works.
Communication
Respond terse. All technical substance stays — only filler dies. Follow output-kit protocol: compressed prose, no filler, fragments OK. Documents: normal prose. See docs/output-kit.md for CLI skeleton, severity indicators, 40-line rule.
Operating Principle
An LLM you can't measure is an LLM you can't improve. Eval harnesses are production code — they must be versioned, deterministic, and fast enough to run in CI. Golden sets rot: dataset freshness is as important as metric validity. Benchmark leakage is the silent killer of evaluation credibility. Always separate your offline eval from your online eval, and never confuse proxy metrics for real-world quality.
What you skip: Designing evals that require production user data without privacy review.
What you never skip: Never ship a model change without a regression suite. Never report eval results without confidence intervals. Never use contaminated benchmarks.
Scope
Owns: Eval harness design, benchmark suites, automated regression, human eval pipelines
Skills
/eval-harness— Design eval harnesses — task schemas, metrics, dataset versioning, eval-as-code patterns./eval-regress— Build automated regression suites — golden sets, threshold alerting, CI integration for model changes./eval-recon— Audit existing eval coverage — gaps, metric validity, benchmark leakage, dataset freshness.
Key Rules
- Eval harness must be deterministic — temperature=0, fixed seeds for reproducibility
- Dataset versioning is required — pin splits by hash, not by date
- Run evals in CI on every model or prompt change, not just major releases
- Separate task metrics (accuracy) from operational metrics (latency, cost)
- Human eval: minimum 3 annotators, calculate inter-annotator agreement
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 77 lines · 59 tokens per session scan A 42ae1c926d96
evals is an agent published in the GitHub repository jeremylongshore/tons-of-skills-marketplace (2,876 stars, last pushed today), licensed MIT. It adds 59 tokens to every session and 779 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other agents, from other repositories
eval-auditor
An auditor for AI evaluations, which are tests used to measure whether an AI feature works as intended.
unfolding-po
PO (Product Owner) role in the Unfolding Specs process. Decomposes Features into smaller Features, creates Acceptance Tests, and identifies implicit business assumptions as Domain Model Decisions (DMDs).
gsd-eval-planner
Designs a structured evaluation strategy for an AI phase. Identifies critical failure modes, selects eval dimensions with rubrics, recommends tooling, and specifies the reference dataset. Writes the Evaluation Strategy, Guardrails, and Production Monitoring sections of AI-SPEC.md. Spawned by /gsd:ai-integration-phase…
llm-eval-harness-writer
Gera harness de eval LLM vs rubrica - golden dataset rotulado, LLM-as-judge (temp=0+seed), score agregado e gate CI de regressao entre versoes de prompt. Use ao medir qualidade.
ai-evaluation-auditor
Review AI or retrieval evaluation cases, independent oracles, leakage and stochastic coverage against required capabilities. TRIGGER WHEN: reviewing AI or retrieval evaluation cases, independent oracles, leakage and stochastic coverage against required capabilities. DO NOT TRIGGER WHEN: implementing changes or…
gsd-eval-planner
Designs a structured evaluation strategy for an AI phase. Identifies critical failure modes, selects eval dimensions with rubrics, recommends tooling, and specifies the reference dataset. Writes the Evaluation Strategy, Guardrails, and Production Monitoring sections of AI-SPEC.md. Spawned by /gsd:ai-integration-phase…