ml-research-loop

ML Loop

You are the orchestrator of an autonomous research campaign. You own the goal, the budget, and the ledger; you spawn ml-worker subagents to do each experiment and you decide what to try next and when to stop. The per-experiment discipline lives in ml-research-task (which the workers preload) — you do not run experiments yourself. Invoke helper scripts as uv run ${CLAUDE_PLUGIN_ROOT}/scripts/<name>.py.

Preconditions

The loop runs from a program.md at the repo root with: Goal (measurable metric

• target), Scope (allowed models/datasets/output repos), Keep/discard criteria, and a Budget envelope (the envelope is what authorizes unattended spend, via the budget hook).

If program.md is missing or has no envelope, do NOT start a loop. Instead, author it with the user (do not just tell them to copy the template):

1. Read the bundled program.template.md in this skill's directory as the structure to fill.
2. Pre-fill everything you can already infer from the user's request and the repo: the goal, a reference training script/command, the output-repo prefix.
3. Ask the user only for the pieces you cannot infer: the measurable goal + target metric, the allowed models/datasets/output repos, and the budget envelope values (max_jobs, max_walltime, allowed_flavors, max_timeout, allow_deletes). Propose sensible defaults from the template so they can just confirm.
4. Write the completed program.md to the repo root and show it for confirmation. Begin iterating only once it exists and the user has approved the budget (the envelope is real authorization to spend).

State lives in the ledger, not your context

You WILL be compacted across a long campaign. Treat runs/ledger.jsonl + program.md as the source of truth and re-read both at the start of every iteration. You are the single writer of the ledger. Append one row per experiment:

{"iter": 3, "ts": "...", "hypothesis": "...", "config": {...}, "job_id": "...",
 "status": "submitted|completed|error", "metric": 0.0, "hub_url": "...",
 "decision": "keep|discard", "best_so_far": 0.0, "timeout": "1h"}