autoresearch

autoresearch

The canonical loop that distinguishes Rockie from a chat assistant. This is what runs around-the-clock on the tenant's Fly machine.

The bones of this skill predate Karpathy's autoresearch (rockie's existing autopilot + queue + post-run-review primitives are richer than Karpathy's narrow loop). Phase 7 augmented those bones with three sharp Karpathy primitives — frozen metric, time-budget per experiment, explicit mutation surface — without stripping the broader system. A later pass added the sustained-campaign layer below (concurrent pipeline staffing, the verdict protocol, subagent tiering, the pre-launch resource/placement red-team, and the novelty re-verification gate) — the operating discipline that keeps a multi-day, multi-experiment campaign honest once a single queue row stops being the unit of work.

When to invoke

• The autopilot daemon invokes this skill on every queue dequeue.
• A researcher invokes manually for a one-off experiment outside the queue (/autoresearch run-once …).
• The post-run-review skill calls back here with a follow-up experiment derived from a [LEARN] block.

The canonical loop

┌─ taste corpus (immutable for the run) ────────────┐
│  SOUL / STYLE / METHODOLOGY / DISMISSALS / MEMORY │
│  + program.md (Karpathy convention) if present    │
└────────────────────┬───────────────────────────────┘
                     │
            ┌────────▼────────────┐
            │ PICK from queue or  │   ← queue-refill skill keeps this stocked
            │ from a [LEARN] tip  │
            └────────┬────────────┘
                     │
                     ▼
       declare mutation_surface[]  ← explicit; every other path is read-only
                     │
                     ▼
         lock metric (metric_locked=1)  ← Karpathy: no goalpost-shifting
                     │
                     ▼
         set time_budget_seconds         ← Karpathy: hard cap, default 300s
                     │
                     ▼
        ┌────────────────────────┐
        │ propose change to ANY  │
        │ file in mutation_surface│   (everything else is immutable)
        └────────────┬────────────┘
                     │
                     ▼
              run the experiment
                     │
                     ▼
        evaluate against the frozen metric
                     │
              ┌──────┴──────┐
        improvement?     no improvement?
              │              │
              ▼              ▼
         git commit     git revert
              │              │
              ▼              ▼
       post-run-review (always)
              │
              ▼
      [LEARN] / [DEAD-END] capture
              │
              ▼
       calibration row (predicted vs actual delta)
              │
              ▼
       [if best-so-far] code-pool admit
              │
              └──→ next iteration