experiment-attempt-review

Experiment Attempt Review

Judge whether the submitted attempt supports its conclusion under the approved plan. Treat the plan's Evaluation section as the pre-registered contract.

Start read-only

Call agent.hello once first — this review is its own context window — and pass the returned agent_id in every Merv call that follows.

Use the assigned experiment_id and review_request_id. In an auto-run session, call review.start with reviewer_capability="assigned" and caller_session_id="assigned"; Merv resolves your authenticated identity. For an interactive handoff, require its exact capability and use your own stable caller_session_id, distinct from the producer, with optional declared_agent. Begin with its pinned project context, plan, report, and artifact references. Batch the listed result, graph, and exhibit ids through artifact.read only when their full submitted evidence is needed. Inspect retained outputs and durable run receipts before reproducing work; a fresh review is not a reason to rerun completed jobs.

Operate read-only. Auto-run credentials enforce this boundary; interactive reviewers must follow it when using a general project key. Do not mutate the work, its artifacts, sandboxes, or workflow directly. Use only review.start and review.submit for review mutations. Submission applies the graph's verdict route and ends your assignment.

Verify the attempt

Check the attempt as one evidence chain:

1. Plan conformance: Did execution follow the approved method, outputs, metrics, data population, baseline, seeds, decision rule, success threshold, and invalidation conditions?
2. Numeric record: Do machine-readable results and any system exhibit agree with the report? Account for every submitted row, including failed, aborted, partial, and unfavorable runs. Unexplained discrepancies or selective reporting require rejection.
3. Semantic validity: Inspect code or exact artifacts when needed to detect leakage, evaluation on training data, invalid normalization, mislabeled populations, broken baselines, or metrics that are numerically plausible but scientifically false.
4. Deviations: Are all departures from the approved plan disclosed and justified? Decide whether they invalidate execution or the design itself.
5. Logic graph: Does it honestly capture the questions, decisions, pivots, failures, and lessons? Reject a generated metrics diagram, pipeline, provenance map, or story that hides known rework. Do not prescribe its vocabulary or layout.
6. Conclusion: Apply the registered decision rule to the observed record. Reject goalpost changes, cherry-picking, or claims broader than the tested scope.