experiment-design-review

Design Review

Review whether the proposed experiment can answer its stated question. Do not improve the plan on the producer's behalf; identify the smallest changes required before execution.

Start read-only

Call agent.hello once first — this review is its own context window — and pass the returned agent_id in every Merv call that follows.

Use the assigned experiment_id and review_request_id. In an auto-run session, call review.start with reviewer_capability="assigned" and caller_session_id="assigned"; Merv resolves your authenticated identity. For an interactive handoff, require its exact capability and use your own stable caller_session_id, distinct from the producer, with optional declared_agent. Use its pinned project_context and experiment context as the default evidence. Read listed artifacts when a load-bearing detail needs deeper inspection. On a revised plan, inspect previous findings and reused evidence before demanding new work; identify what evidence would disprove the claim and whether the plan can produce it.

Operate read-only. Auto-run credentials enforce this boundary; interactive reviewers must follow it when using a general project key. Do not mutate the work, its artifacts, sandboxes, or workflow directly. Use only review.start and review.submit for review mutations. Submission applies the graph's verdict route and ends your assignment.

Judge the design

The server already checks that required headings exist. Judge whether their content is scientifically sufficient:

• Summary: Does a human reader understand what will be tested and why?
• The ask: Does the plan answer the experiment's intent? Where the creator supplied details, does the plan engage them — adopting each point or stating why not? A plan that tests something adjacent to the intent is a send-back.
• Objective and hypothesis: Is the claim explicit and scoped? Is the expected direction and motivation clear?
• Evaluation: Are the metrics, comparison or baseline, decision rule, success threshold, and invalidation conditions concrete and appropriate? Would meeting them actually justify the proposed conclusion?
• Method: Is the procedure executable, appropriately sized, and capable of isolating the claim?
• Right-sizing: Is this the smallest credible experiment that can produce a decision-relevant signal about the intent? Are additional data, scale, variants, seeds, compute, or infrastructure justified by a necessary distinction or known validity risk?
• Prior work and provenance: When earlier project findings or research papers materially inspire the design, does the plan cite the relevant exp_... experiments and use portable, source-native paper references (arXiv id, DOI, or stable canonical source URL), while identifying what is being carried forward? An internal paper_... id does not satisfy this check. Omission is acceptable when the design is genuinely independent.
• Outputs: Are the evidence files that must survive execution named?
• Risks and confounders: Are material failure modes, leakage risks, and alternative explanations addressed?