paper-researcher

paper-researcher is an agent for coding agents from mxslr/mlcraft. It costs 87 tokens per session (534 once invoked), scanned A, original, MIT.

A read-only research agent for finding and critically comparing academic papers and benchmarks for machine-learning and artificial-intelligence tasks. It produces a structured summary of methods, datasets, metrics, and realistic performance limits.

In plain words
What is it for?
Use it to search papers, compare methods and datasets, check how data was split, extract reported metrics, and estimate a realistic benchmark ceiling.
Why use it?
It helps separate reliable results from misleading numbers caused by data leakage, weak data splits, small test sets, or easier cropped-image tasks.

Agent

Part of the mlcraft plugin — 23 skills, 1 command, 1 agent shipped together

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/mxslr/mlcraft/paper-researcher
Clone the repo
git clone --depth 1 https://github.com/mxslr/mlcraft

Or install mlcraft, the plugin that ships this one along with the rest of its 23 skills, 1 command, 1 agent.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for paper-researcher

README.md
[![agentmods](https://agentmods.dev/badge/agents/mxslr/mlcraft/paper-researcher.svg)](https://agentmods.dev/agents/mxslr/mlcraft/paper-researcher)
Your own site
<a href="https://agentmods.dev/agents/mxslr/mlcraft/paper-researcher"><img src="https://agentmods.dev/badge/agents/mxslr/mlcraft/paper-researcher.svg" alt="Measured on agentmods" height="20"></a>
Per session 87 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 534 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00087 $0.00534
Opus 5 $0.00044 $0.00267
Sonnet 5 $0.00017 $0.00107
Haiku 4.5 $0.00009 $0.00053

Measured 5d ago against content hash 26e70982078f, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-05, from the pricing page.

Security

Grade A, and why

paper-researcher scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

agents/paper-researcher.md · 29 lines

What it actually says

You are a meticulous ML research librarian. Your job is to produce an evidence-grounded literature brief for a given task/dataset - not to write code.

Method

  1. Run several diverse web searches (by task, by dataset name + "benchmark/state of the art", by method family, by "leakage" / "patient-level split"). Prefer arXiv, Nature/Springer, PubMed, PMC, Papers-with-Code, reputable venues.
  2. Fetch the most relevant 2-5 sources and read them. Extract: method/architecture, dataset + split used, headline metric, and how the split was done.
  3. Be adversarial about numbers. For each strong claim ask:
  • Is this on cropped ROI / patches or the whole image? (ROI is a different, easier problem.)
  • Is the split per-group / per-patient / temporal, or random per-sample (leakage)?
  • Is the test set tiny? Is the metric thresholded accuracy (unstable) vs AUC?
  • Is it the official benchmark split (comparable) or a custom one?
  1. Establish a realistic ceiling: the best number from rigorous, leakage-free, comparable studies - not the highest number anywhere.

Output (return this, nothing else)

  • Task framing: problem type, standard dataset(s), the metric that matters.
  • SOTA table with columns: method, dataset/split, metric, and a 1-line credibility note (rigorous / likely leaky / ROI-only).
  • Realistic ceiling: the honest number to aim for, with the source.
  • Top techniques worth combining (ranked), each with the evidence and the caveat.
  • Traps: leakage sources, preprocessing gotchas, common inflated claims to distrust.
  • Sources: markdown links to everything you used.

Keep it concise and factual. Flag uncertainty explicitly. Never invent citations or numbers.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 5d ago First seen · 29 lines · 87 tokens per session scan A 26e70982078f

Subscribe to this mod's changes

paper-researcher is an agent published in the GitHub repository mxslr/mlcraft (8 stars, last pushed 1mo ago), licensed MIT. It adds 87 tokens to every session and 534 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other agents, from other repositories

rl-debugger

Use this agent to diagnose reinforcement learning training failures: flat or collapsing reward curves, policy entropy collapse, value-function problems, NaN losses, high-variance runs, and suspected environment bugs. It encodes the RL debugging playbook from the reinforcement-learning skill — pre-training sanity…

Aznatkoiny/zAI-Skills · 312 tokens

surveyor

Research-landscape survey for a problem in theoretical physics or mathematics. Reads problem.md and runs web/arxiv search; produces survey.md with background, known approaches, pitfalls, and key references. May fetch papers into references/.

huggingface/physics-intern-claude-plugin · 47 tokens

demand-generation

Demand Generation (CMO). Owns plugins/demand-generation/ and nothing else. Delegate work in this department's remit here.

cbrock84/headcount · 30 tokens

optimizer

Performance specialist. MUST BE USED for performance issues, database design, query optimization. PROACTIVELY handles profiling, schema design, migrations, and bottleneck identification.

komluk/scaffolding · 35 tokens

ai-visibility-analyzer

Deep autonomous brand visibility analysis across all 6 AI engines — multi-pass real-query probes (5+ engines × 5–10 queries), business-DB scan, citation source enumeration, competitor disambiguation, full 30-day defense plan. Use ONLY when the user explicitly asks for "deep AI visibility analysis", "agent mode"…

akii-technologies-ltd/akii-seo-ai-search-optimizer · 155 tokens

metadata-extractor

Extracts paper metadata (authors, date, venue, fields, DOI/arxiv ID) and a paper-quality assessment (credibility, experimental rigor, reproducibility) from a paper's plain text. Invoked alongside lite-drafter and finding-extractor during /paperloom:ingest.

trapoom555/claude-paperloom · 63 tokens