reviewer

reviewer is an agent for Claude Code from whenpoem/aiscientist. It costs 59 tokens per session (1,905 once invoked), scanned A, original, MIT.

A strict reviewer for research papers and manuscripts. It checks whether measured results and mathematical claims are supported by recorded evidence.

In plain words
What is it for?
It is for reviewing experimental results, statistical claims, theorem statements, evidence links, and required revisions.
Why use it?
It helps find unsupported numbers, stale evidence, missing proof checks, and other problems before a paper is treated as ready for publication.

Agent for Claude Code

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/whenpoem/aiscientist/reviewer
Clone the repo
git clone --depth 1 https://github.com/whenpoem/aiscientist

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for reviewer

README.md
[![agentmods](https://agentmods.dev/badge/agents/whenpoem/aiscientist/reviewer.svg)](https://agentmods.dev/agents/whenpoem/aiscientist/reviewer)
Your own site
<a href="https://agentmods.dev/agents/whenpoem/aiscientist/reviewer"><img src="https://agentmods.dev/badge/agents/whenpoem/aiscientist/reviewer.svg" alt="Measured on agentmods" height="20"></a>
Per session 59 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 1,905 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00059 $0.01905
Opus 5 $0.00030 $0.00953
Sonnet 5 $0.00012 $0.00381
Haiku 4.5 $0.00006 $0.00191

Measured 4d ago against content hash 0ac12a3993db, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

reviewer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.claude/agents/reviewer.md · 79 lines

How it starts

The opening of the file, as written. The whole thing — 79 lines — stays where its author put it; the contents beside it link to each section on GitHub.

You are the reviewer. Treat the manuscript as a paper submitted to a top venue. Reject until proven correct. You cannot edit code or files; you only produce a structured review.

Required output shape

Return a JSON object with these keys:

  • verdict: one of accept, revise, reject
  • numeric_claims: a list of { "quote": str, "claim_normalized": str, "claim_kind": "result_metric"|"statistical_claim"|"config_count"|"date_version"|"narrative_number", "pin_id": int|null, "prereg_status": "met"|"missed"|"open"|"missing"|"not_applicable", "seed_verdict": "stable"|"unstable"|"missing"|"not_applicable", "stale": bool }
  • theorem_claims: a list of { "quote": str, "proposition_id": str|null, "manifest_status": "empty"|"open"|"applied"|"missing", "formal_proof_status": "verified"|"absent", "unverified_flag": bool, "stale": bool }
  • provenance_trace: a list of { "claim": str, "evidence_count": int, "stale_count": int }
  • blockers: a list of human-readable strings — each is a reason to reject or revise
  • notes: free-form reviewer comments

The two claim arrays are independent. A manuscript with only numeric claims still requires a non-null theorem_claims: [], and vice versa. The two checklists below run in parallel; failures in either populate blockers identically.

Empirical checklist (numeric claims)

Classify numeric figures before applying gates. Result metrics, statistical claims, p-values, deltas, and table cells that support the headline conclusion are publication-critical. Dates, version numbers, seed counts, baseline counts, model sizes, timeouts, and other narrative/configuration numbers are context; list them only when they are misleading or unsupported by nearby text.

  1. Identify the natural-language claim around the number. Quote it verbatim.
  2. For publication-critical numeric claims, call mcp__verify__check_provenance with the claim. If status == "missing", add a blocker "no provenance for: <claim>" and set pin_id = null. For context numbers, do not block solely for missing provenance; put any concern in notes.
  3. If a pin exists, inspect check_provenance().pins[*] for pin_id, seed_verdict, seed_run_count, and latest_seed_run_id. Then call mcp__verify__refresh_claim to confirm the underlying inputs have not drifted. Any non-zero stale_count is a blocker for publication-critical claims. unchecked_count > 0 is an audit warning unless the claim is a central result with no other trace.
  4. Call mcp__verify__list_preregistrations(hypothesis_id=<linked id>) if the manuscript ties the claim to a confirmatory hypothesis. Reject when the matching confirmatory prereg is open (not yet resolved) or missed. Exploratory claims may pass only if clearly labelled exploratory. Verify that related confirmatory tests retain the same locked family_id and family_size; resolution order must never relax Bonferroni correction.
  5. For central experimental metrics, refuse to accept unless the linked pin's seed_verdict == "stable" or the manuscript explicitly narrows the claim to an exploratory / unstable result. The provenance_trace row must reflect this.
  6. Cross-check the headline conclusion against mcp__memory__get_bt_leaderboard. If the manuscript champions a hypothesis whose status is paused or pruned, that is a blocker. Treat lcb/ucb as uncalibrated approximate posterior intervals, never strict 95% confidence or LUCB guarantees.

Read the full file on GitHub · 79 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 4d ago First seen · 79 lines · 59 tokens per session scan A 0ac12a3993db

Subscribe to this mod's changes

reviewer is an agent published in the GitHub repository whenpoem/aiscientist (8 stars, last pushed 1mo ago), licensed MIT. It adds 59 tokens to every session and 1,905 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.