scientific-figure

Scientific Figure Loop

The artifact is a scientific figure (the rendered image + the plot.py that produces it). Each iteration generates → critiques+grades: a generator authors a rendering script and renders the figure to land the frozen <goals> message; an adversarial critic grades it 0-100 against the fixed rubrics/rubric.md and decides pass; the generator then revises against the critic's concrete findings. The loop runs until the grade clears <pass_threshold> or the budget is hit. All work happens on copies inside a sandbox; the user's data is copied in read-only and never edited.

The cast (all in this folder):

• roles/generator.md — drafts/revises plot.py, renders figure.png by running <render_cmd>, optionally grounds domain content via <lit>; writes generation_notes.md.
• roles/critic.md — the adversarial grader: re-derives each rubric axis independently, spot-checks the figure's numbers against the data, optionally lit-checks domain completeness, and emits schemas/critique.schema.json (the grade + pass + executable findings).
• rubrics/rubric.md — the fixed grading rubric (the critic never edits it).
• schemas/critique.schema.json — the one validated output.

Spawn-or-degrade. On Claude Code, spawn the generator then the critic as real Agent subagents (sequential — the critic needs the generator's figure); otherwise adopt each role inline. You are the orchestrator.

Why the critic grades itself (the honesty problem)

The critic both critiques and grades, which under loop-termination pressure invites inflation and a generator that games the rubric. roles/critic.md + rubrics/rubric.md counter this: the critic (1) applies a fixed rubric it never edits, (2) re-derives each axis from the rendered figure + data + frozen <goals> rather than echoing the generator, (3) recomputes a sample of the figure's numbers itself instead of trusting "it's fixed", (4) holds a fixed, anchored bar with no credit for effort or elapsed iterations, and (5) applies hard gates (a figure value that contradicts the data, a misleading axis, or fabricated data presented as real fails the figure regardless of the average). The generator optimizes the concrete findings; the critic grades holistically against the frozen goal — so "address every finding" does not mechanically buy a pass. Because the two are separate agents, the critic never just rubber-stamps the generator's intent.