auto

Auto Pipeline: Idea → Experiments → Verify → Review

End-to-end autonomous run for: $ARGUMENTS (when provided) or the project's task.md (when $ARGUMENTS is empty).

Each stage runs in a dedicated agent (isolated context + configurable model). The orchestrator only sees each agent's final summary and the files on disk — it uses those to fire the gates between stages.

⏰ First action — register the hourly notification timer. Before any pipeline work, if task.md opts into notifications (see the Notifications rule in Key Rules), register a recurring scheduled task with CronCreate (cron <off-minute> * * * * — one call per hour, pick a minute ≠ 0/30; prompt: /notify hourly). This fires the hourly briefing on a real wall-clock timer instead of relying on the orchestrator remembering to poll during long waits. The timer is the sole source of the hourly cadence — the orchestrator never separately polls or manually fires /notify hourly. Register it once, up front — first CronList to check an equivalent /notify hourly job is not already scheduled (e.g. on resume), and skip if so. When task.md does not opt into notifications, skip this entirely (no timer). The timer covers only the hourly cadence; the event-driven /notify touchpoints (progress / done / halted / approval-needed) are separate and stay orchestrator-initiated (see Key Rules).

Direction source resolution:

• $ARGUMENTS non-empty → use it as the research direction; if task.md also exists, the claim agent treats task.md as authoritative detailed context (current behavior).
• $ARGUMENTS empty + task.md exists → use task.md as the sole direction source; pass direction: "" to the claim agent so it relies entirely on task.md.
• $ARGUMENTS empty + task.md absent → stop and report [direction] no input — provide $ARGUMENTS or create task.md. Do not invent a direction.

Defaults

Flag	Default	Effect
AUTO_PROCEED	true	When true, gates skip the UI prompt entirely and directly pick the recommended option. When false, the orchestrator calls AskUserQuestion and waits for the user.
RESUME	false	When true, the orchestrator skips any stage whose final artifacts already exist non-empty on disk, and forwards resume: true to each invoked agent so sub-skills can do phase-level skipping too. Useful for picking up after a crash, or for re-running only the missing stages. Default false = always run every stage from scratch (and overwrite previous artifacts).
REVIEW_LOOP	true	Run iteration after verify; set false to stop at verify.
MODEL	none	Global model family alias applied to every stage whose <STAGE>_MODEL is unset. Accepts opus / sonnet / haiku (case-insensitive). Per-stage <STAGE>_MODEL always wins over this. When both are unset, the stage uses its frontmatter model: line if present, else inherits the session model (claude --model …).
<STAGE>_MODEL	none	Per-stage model family override for any of CLAIM, EXPERIMENT, VERIFY, ITERATION. Accepts only family aliases: opus, sonnet, haiku (case-insensitive). When unset, falls back to the global MODEL, then the agent's frontmatter model: line if present, else the session model (claude --model …). Pinning lives only in agents/<name>.md frontmatter — add a model: line to pin a stage, leave it absent to inherit the session model. CLI/frontmatter aliases resolve to the family's latest version (opus → newest opus, not claude-opus-4-x), so running a stage at exactly the session model requires an absent frontmatter model:.
DIMENSIONS	model	Verify swap axes — and therefore the variant count per picked claim, since verify runs exactly one swap per listed axis. List or comma-separated subset of {method, dataset, model}. Default model → 1 variant/picked-claim (fast, single-axis model swap). Broaden with dimensions: method,dataset,model → 3 variants/picked-claim (full stress test). Forwarded to verify agent.
TARGET_CLAIMS	all	Which claims verify stress-tests: all (default; covers both main-experiment-supported and main-experiment-rejected claims so robustness is checked in both directions) / passed (only main-experiment-supported = claim_supported = pass) / failed (only main-experiment-rejected = claim_supported = fail) / a specific claim id. Note passed ∪ failed = all. Forwarded to verify agent.
MAX_VERIFY_CLAIMS	1	Cap on how many Stage-1-admitted claims proceed into Stage 2 (swap variants). Stage 1 (main-experiment integrity audit) always audits every target claim regardless — the cap only gates Stage 2 entry. When the admitted pool exceeds the cap, /auto-verify's Phase 3 step 0 picks the top-K by importance judgment (reading each admitted claim's statement against upstream narrative like IDEA_REPORT.md / ## Rationale; row order is NOT a priority signal). Un-picked admitted claims are marked INTEGRITY_ONLY with stage2_skip_reason: max_verify_claims_cap in VERIFY_REPORT.md; user can swap-test them later via /auto-verify <id> — resume: true (Stage 1 audit is reused via RESUME). Forwarded to verify agent. At the default cap of 1 with default DIMENSIONS=model, verify launches 1 × 1 = 1 variant run per /auto pass.
ROBUSTNESS_THRESHOLD	0.5	A claim PASSes verify iff robustness ≥ ROBUSTNESS_THRESHOLD. Set higher (e.g., 0.67) for stricter publication-ready verification, lower (0.33) for exploratory work. Forwarded to verify agent.
MIN_VARIANTS_FOR_VERDICT	1	Minimum number of integrity-clean variants required to issue a PASS / FAIL verdict on a claim. Default 1 means a single eligible variant still yields a verdict; N_eligible < MIN_VARIANTS_FOR_VERDICT triggers ZERO_ELIGIBLE_VARIANTS (a distinct terminal state from INCONCLUSIVE — see auto-verify/SKILL.md Phase 10). Raise to 2/3 for stricter projects where you want multiple independent axes to agree before issuing PASS / FAIL. Forwarded to verify agent.
BASE_REPO	none	GitHub repo URL to clone before implementing.
RESEARCH_DOMAIN	auto	Project domain tag (e.g. mechanistic-interpretability, vision-encoders, rl-policy-eval). Used to gate mechanism-family routing and downstream auxiliary decisions. Default auto: orchestrator forwards auto to the experiment agent and the sub-skill infers from FINAL_PROPOSAL.md; if inference is ambiguous, silently falls back to general regardless of AUTO_PROCEED (no UI prompt). To force a specific domain, set this flag explicitly on the CLI.
COMPACT	false	Generate compact summary artifacts: idea-stage/IDEA_CANDIDATES.md after claim, refine-logs/EXPERIMENT_LOG.md after experiment, skip per-claim verify/<claim_dir>/ROBUSTNESS.md (where <claim_dir> = <claim_id>_<short_claim> on disk; see skills/auto-verify/SKILL.md "Directory Layout"). Forwarded to claim / experiment / verify agents.
CODE_REVIEW	true	External LLM reviewer checks experiment + verify-variant code before deployment. Set false to skip. Forwarded to experiment + verify agents.
SANITY_FIRST	true	Run the smallest/cheapest run first to catch setup bugs before launching the full suite. Forwarded to experiment + verify agents.
AUTO_DEPLOY	true	Auto-deploy after implementation + review; set false to pause for manual inspection. Forwarded to experiment + verify agents.
MAX_PARALLEL_RUNS	4	Max concurrent /run-experiment calls dispatched within the experiment / verify stages. Becomes max_parallel: in the /experiment-queue manifest when Phase 4 of experiment routes to the queue path. Forwarded to experiment + verify agents.
BATCH_DISPATCH	auto	Phase 4 dispatch routing rule (experiment stage). auto (default) lets /auto-experiment auto-pick /run-experiment for small milestones (≤ 5 ad-hoc runs) and /experiment-queue for large or dependency-laden ones (≥ 10 runs, depends_on, grid expansions, ≥ 3-seed × ≥ 3-config sweeps). queue forces every milestone to /experiment-queue; direct forces every milestone to /run-experiment (debug-only — emits a warning if it overrides the queue rule). Forwarded to experiment agent.
REF_PAPER	false	Reference paper for the claim stage to summarize first. Accepts a local PDF path, an arXiv abs URL, or any paper URL. When set, claim Phase 0.5 writes idea-stage/REF_PAPER_SUMMARY.md and downstream idea generation builds on it. Forwarded to claim agent as ref_paper:.
BEHAVIOR_SOURCE	given	Behavior stage — controls where the behavior to study comes from and whether it is validated. Three values: given (default): the behavior is already specified in the direction / task.md and assumed to hold — no ideation, no novelty check, no M0 validation; the claim stage faithfully captures it and goes straight to the mechanism. Precondition enforced by the Given-Behavior Comprehension Gate (see the claim stage): task.md must name a concrete behavior; if it names only a topic, the orchestrator asks the user to specify a behavior or switch to discovery (this gate always waits — even in full-auto). given-validation: the behavior is given the same way (faithfully captured from task.md, no mining, no ideation, no novelty) but its existence is validated first — the experiment plan opens with a hard M0 phenomenon-validation gate that the experiment stage runs before any mechanism compute. discovery: the behavior itself is mined — load /mechanism-behavior-discovery to sharpen a new candidate phenomenon and run the full ideation pipeline (research-lit → idea-creator → novelty-check → impact-check → research-review → research-refine-pipeline; final idea chosen by impact-first, novelty-second), then the experiment plan opens with the M0 gate too. The M0 four-state verdict (established/conditional proceed; not-established ends the pipeline with a negative-result report skipping verify+iteration; inconclusive re-runs M0) applies to both given-validation and discovery. Accepts given / given-validation / discovery (case-insensitive). Forwarded to the claim agent as behavior_source:.
MECHANISM	discovery	Mechanism stage — controls who picks the mechanism method. discovery (default): the system selects the mechanism family — the experiment stage runs /mechanism-skills routing (route_only → auto-select the recommended candidate, or the family mini-prompt when AUTO_PROCEED=false) and the claim stage loads /mechanism-explore to shape the hypothesis direction + experiment plan. given: the user has specified the mechanism method/family in task.md — the claim stage captures it and the experiment stage commits it directly (Phase 1.5 Mode B, CHOSEN_FAMILY=<the method>), bypassing routing and the family mini-prompt; task.md must name a concrete mechanism method/family, else the claim stage halts (report it back). Accepts given / discovery (case-insensitive). Forwarded to the claim agent as mechanism:. Resource fidelity: the claim stage stamps resource_fidelity: strict into FINAL_PROPOSAL.md + EXPERIMENT_PLAN.md iff BEHAVIOR_SOURCE=given AND MECHANISM=given (the reproduction combination) — the experiment stage then enforces the Resource-Fidelity Harness on the main experiment (exact models/datasets at full scale — no smaller-model swap, no data subsetting, no skipped must-run runs; OOM handled by batch↓/grad-accum/sharding/offload, HALT rather than downscale; verify is exempt, its swaps are intentional). Every other combination leaves the marker unstamped, so model/dataset choice is unconstrained (cost-aware).
MAX_ITERATIONS	6	Iteration agent: max total back-edge actions (variant fix / plan+script fix / claim-stage re-entry) before stopping. Counts every back-edge uniformly; new claims produced by claim-stage re-entry inherit the same budget (no fresh allocation). Forwarded to iteration agent. Legacy alias MAX_ROUNDS is accepted at the CLI for one release and silently normalized to MAX_ITERATIONS.
MAX_CLAIM_REENTRIES	2	Iteration agent: sub-budget within MAX_ITERATIONS for claim-stage re-entries (action type ③ — see auto-iteration-loop/SKILL.md). Prevents the failure mode where the reviewer keeps requesting claim rewrites without anyone fixing the experiments behind them. When exhausted, the iteration loop refuses further ③ actions even if iterations remain. Forwarded to iteration agent.
TARGET_SCORE	6	Iteration agent: stop early when score ≥ this AND verdict is ready/almost AND no FAIL / INCONCLUSIVE / ZERO_ELIGIBLE_VARIANTS claim remains (three-dimensional STOP rule).
GPU_ID	auto	GPU device(s) to pin every experiment / verify-variant / iteration-round run to. auto lets each sub-skill inherit from the environment / launcher (no pinning). A single id (0) or comma-list (4,5,6,7) causes the sub-skill to pass CUDA_VISIBLE_DEVICES=<GPU_ID> as the first positional argument to /run-experiment (and sanity runs); /run-experiment is a Skill, not a shell command, so it cannot accept a shell prefix — instead it parses this positional arg and exports CUDA_VISIBLE_DEVICES itself before launching the experiment subprocess. Forwarded to experiment + verify + iteration agents (iteration's Phase-C /run-experiment dispatches inherit the same pin).
OOM_MAX_GPUS	4	resource_fidelity: strict only (the BEHAVIOR_SOURCE=given + MECHANISM=given reproduction combo). Upper bound on how many free GPUs the Resource-Fidelity Harness may auto-add to a single OOM'd run before halting. On OOM the harness queries free GPUs (memory.used < 500 MiB), auto-adds them to the run and enables sharding — auto-converting a single-GPU script (model.cuda() / .to(device)) to device_map="auto" / FSDP / CPU-or-disk offload, first verified numerically equivalent on a fit-on-one-GPU proxy slice — repeating up to this cap while leaving MAX_PARALLEL_RUNS headroom for sibling runs. Only after the cap is hit (or no free GPU remains) and CPU/disk offload is exhausted does it HALT — and never by downscaling (smaller model / subset data stay forbidden, even in full-auto). Set higher to let one big run spread across more devices; auto lets it use all currently-free GPUs. Forwarded to experiment + verify + iteration agents. Ignored when the strict marker is absent (every non-reproduction combination).
UNDERPOWER	tag	Active whenever the strict marker is absent (ignored under resource_fidelity: strict — the reproduction combo — which already forbids subsetting). Guards against an under-powered cheap run's weak/negative verdict being mistaken for a real negative. After the experiment, a claim whose main-experiment verdict is weak (not-supported / partial / null) and whose realized scale is materially below the plan (used_n shortfall, fewer seeds, or fewer grid/checkpoint points than EXPERIMENT_PLAN.md) is flagged suspected under-power. tag (default): tag the claim [suspected under-power: used_n X/Y, seeds A/B, grid P/Q] (recorded as a provisional caveat carried into the ledger + verify + iteration, so the negative is treated as provisional, not a confirmed falsification) and proceed — respecting the cost-aware design; under AUTO_PROCEED=false the experiment agent instead asks (full re-run / targeted-milestone re-run / accept demo-scale). stop: treat a suspected-under-power claim as a Round-End Decision (ended-needs-decision (experiment: suspected-under-power)) even under AUTO_PROCEED=true, so you decide whether to re-run at full scale before verify trusts it. off: disable the check. Forwarded to experiment agent.
LEDGER_FIGURES	auto	Whether the final ledger render should call /paper-figure to produce per-claim plots and tables embedded into CLAIMS_LEDGER.md (image figures = PNG inline + PDF link; tables = Markdown inline + .tex link). Tri-state: auto (default) generates figures only for claims with at least one plottable or tabulable data source; true forces an attempt for every non-deferred claim (an unsupported claim still degrades to a skipped entry, never a halt); false disables the hook entirely and the ledger renders without a Figures bullet. Fires once per pipeline run, at the final ledger hook only (iteration:final, or verify when REVIEW_LOOP=false); intermediate hooks never re-invoke /paper-figure. See Ledger Figures hook.