Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add richfrem/agent-plugins-skills --skill eval-autoresearch-fitgit clone --depth 1 https://github.com/richfrem/agent-plugins-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/richfrem/agent-plugins-skills/eval-autoresearch-fit)<a href="https://agentmods.dev/skills/richfrem/agent-plugins-skills/eval-autoresearch-fit"><img src="https://agentmods.dev/badge/skills/richfrem/agent-plugins-skills/eval-autoresearch-fit/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/richfrem/agent-plugins-skills/eval-autoresearch-fit"><img src="https://agentmods.dev/badge/skills/richfrem/agent-plugins-skills/eval-autoresearch-fit.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00085 | $0.02664 |
| Opus 5 | $0.00043 | $0.01332 |
| Sonnet 5 | $0.00017 | $0.00533 |
| Haiku 4.5 | $0.00009 | $0.00266 |
Grade A, and why
eval-autoresearch-fit scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 258 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Evaluate Autoresearch Fit
Assess whether a skill is a viable candidate for the Karpathy 3-File Autoresearch autonomous
optimization loop. Scores each skill on four dimensions, proposes what the 3-file architecture
would look like, and updates the canonical summary-ranked-skills.json via the update script.
Background
The Karpathy autoresearch pattern requires three conditions simultaneously:
- A Clear Metric — a single number with a clear optimization direction
- Automated Evaluation — no human in the loop; scoring runs headlessly from a shell command
- One Editable File — the agent mutates only a single predefined target per loop
Skills that lack these properties cannot run an effective autonomous loop.
Data File
The canonical ranked skills list lives at:
plugin-research/experiments/analyze-candidates-for-auto-reseaarch/skills/eval-autoresearch-fit/assets/resources/summary-ranked-skills.json
After every evaluation, update it with the update script (see Step 5).
Scoring Dimensions
Each dimension is scored 1-10. Max total = 40.
| Dimension | 10 (Best) | 1 (Worst) |
|---|---|---|
| Objectivity | Binary pass/fail or exact numeric output from a shell command | Purely subjective, requires human taste judgment |
| Execution Speed | Completes in seconds | Requires 30+ min or human input |
| Frequency of Use | Triggered multiple times per day | Rarely needed (monthly or less) |
| Potential Utility | Prevents systemic failures or saves hours per session | Nice-to-have improvement |
Viability thresholds:
- 32-40 HIGH — Excellent candidate, implement now
- 24-31 MEDIUM — Good candidate, address identified gaps first
- 16-23 LOW — Needs significant rework to be viable
- < 16 NOT_VIABLE — Skip or the metric is unfixable
Evaluation Steps
Step 1: Locate the Skill
If $ARGUMENTS is a path to a directory containing SKILL.md, read it directly.
Otherwise find it by name from the repo root:
PROJECT_ROOT=$(git rev-parse --show-toplevel)
find "$PROJECT_ROOT/plugins" -name "SKILL.md" | grep "$ARGUMENTS" | head -5
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago First seen · 258 lines · 85 tokens per session scan A 6cd15a53100c
eval-autoresearch-fit is a skill published in the GitHub repository richfrem/agent-plugins-skills (6 stars, last pushed today), licensed MIT. It adds 85 tokens to every session and 2,664 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
composio
Use Composio from Agent Swarm through the agent-swarm x composio CLI route, the swarmx MCP tool, or a registered ctx.api.composio script connection. Trigger when a task needs connected third-party app tools such as Gmail, Google Calendar, Google Docs, Google Drive, GitHub, Slack, Notion, or HubSpot through Tool Router…
close-issue
Close a GitHub or GitLab issue with a summary comment.
soul
Embody this digital identity. Read SOUL.md first, then STYLE.md, then examples/. Become the person—opinions, voice, worldview.
implement-issue
Implement a GitHub issue or GitLab issue and create a PR/MR.
bbc-skill
A read-only tool for collecting all comments from a Bilibili video, including replies and pinned comments. Bilibili is a Chinese video-sharing platform.
feature-implementation
Implement software features, fix bugs, and write production-quality code. Use this for hands-on development work including writing new code, modifying existing code, and debugging issues.