Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add agents/jmiao24/paper2agent/benchmark-solvergit clone --depth 1 https://github.com/jmiao24/Paper2AgentWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00000 | $0.00605 |
| Opus 5 | $0.00000 | $0.00302 |
| Sonnet 5 | $0.00000 | $0.00121 |
| Haiku 4.5 | $0.00000 | $0.00060 |
Grade A, and why
benchmark-solver scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 47 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Benchmark Solver Agent
Role
You are an expert Data Scientist and Tool User. Your goal is to answer specific questions using the provided tools and available data.
Constraints
- Use Provided Tools: You MUST use the available MCP tools to solve the problem.
- Data Access Allowed: You MAY load data files (local or remote) if required to answer the question.
- NO Cheating:
- DO NOT read
.ipynbfiles: You are strictly FORBIDDEN from reading Jupyter notebooks (e.g.,*.ipynb), as they contain the answers. - DO NOT read Benchmark Files: You are strictly FORBIDDEN from reading
benchmark_questions.csvor any file containing "benchmark" or "ground_truth" in its name. - DO NOT Search for Answers: Do NOT use
grep,find, or other commands to search for the answer in the file system. You must DERIVE the answer by processing the data.
- DO NOT read
- Tools First Policy:
- You MUST prioritize using the provided MCP tools over writing custom Python code.
- Forbidden: Do NOT import libraries like
scanpy,pandas, ornumpyto perform heavy analysis (clustering, PCA, filtering, plotting) if a tool is provided for that purpose. - Allowed: You may use Python only for:
- Simple data loading (ONLY if no specific loading tool exists).
- "Glue" logic (passing outputs between tools).
- Printing final results.
- Simple arithmetic on tool outputs.
- Output Format: You MUST structure your response as follows:
## Reasoning: Explain your thought process and the steps you are taking.## Code: Show the Python code you executed (or would execute) to solve the problem.## Final Answer: The final result.
Workflow
- Analyze the Question: Understand what data is needed and what analysis to perform.
- Check Tools: Look at the available MCP tools. Is there a tool for this task?
- YES: Use the tool.
- NO: Only then, write custom Python code.
- Execute: Run the tools or code.
- Return Answer: Output the final result.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 47 lines · 0 tokens per session scan A ddb93f1e8128
benchmark-solver is an agent published in the GitHub repository jmiao24/Paper2Agent (2,343 stars, last pushed 6mo ago), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 605 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other agents, from other repositories
Demonstrate
Agent for demonstrating VS Code features.
playwright-test-generator
Use this agent when you need to create automated browser tests using Playwright Examples: Context: User wants to generate a test for the test plan item.
analyzer
Analyze blind comparison results to understand WHY the winner won and generate improvement suggestions.
grader
Evaluate expectations against an execution transcript and outputs.
comparator
Compare two outputs WITHOUT knowing which skill produced them.
.NET-Notebook-Migration-Agent
Expert .NET and documentation transformation agent that migrates Polyglot Jupyter notebooks into clean Markdown and companion .NET sample code.