dueling-autoresearch

Dueling Autoresearch Loop

Two lanes work the same objective in parallel and race the same metric — by default a classical/algorithmic lane against an ML/learned lane (the lanes are user-named). Each lane runs its own analysis-first iteration via roles/TrackAgent.md, confined to its lane. Every round both lanes post to a shared duel_log.md scoreboard and may borrow ideas across the lane boundary — but each stays in its lane. The feedback signal is the shared <metric> on a shared eval: if the classical lane wins, that is a real result. Lanes support mixed code locations — a codebase lane edits existing repo files, a sandbox lane authors its own code — and an eval-parity gate keeps the scores comparable.

You are the orchestrator: each round you advance both lanes, update the scoreboard, and keep both honest. Do not pause for permission once the loop is running.

When to use

Use this to race two genuinely different approaches on one metric and keep them honest against the same eval — classical vs learned, two model families, two query strategies. Default to spawning both lanes in parallel and letting the scoreboard drive cross-lane idea borrowing; if a lane runs dry, push it to a more radical in-lane change or to borrow a fresh idea from the log. Not for tuning a single approach (use a single-track loop), and not for a one-shot comparison of two finished things.

The cast (all in this folder):

• roles/TrackAgent.md — the per-lane researcher, instantiated once per lane.

Setup

Resolve bindings interactively. If loop.run.yaml exists in the working dir, load it, confirm the values in one line, and skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write loop.run.yaml (format: examples/run.example.yaml) and confirm the values before creating any other files.