os-lab-loop

Lab Loop (os-lab-loop)

The single loop that improves a target skill or agent in the lab. It owns orientation, mutation and close. It never scores: each change goes to os-lab-scorer.

Contents

• Critical constraints
• Quick start
• Workflow
• Verification
• References

Critical constraints

1. Every change is scored: call os-lab-scorer for each mutation; no manual rollback and no self-assessed KEEP.
2. Locked evaluator: only the target SKILL.md (and, as a second-order move, the proposer prompt or references/program.md) is mutated; never evaluate.py or evals/evals.json.
3. One variable per iteration, and a documented hypothesis before the first mutation.
4. Session close is mandatory: surveys, ledger, metrics, registry and memory run even when the target score is not reached.
5. Budget guard: 10 iterations per session, 3 consecutive DISCARDs trigger a strategy review (see lab-limits).
6. Do not skip orientation: no agent starts cold.

Quick start

From a lab prepared by os-lab-setup, with a baseline already recorded in <path/to/target-skill>/evals/results.tsv:

Run the improvement loop on <path/to/target-skill> for 5 iterations.

Each iteration: classify the failure, propose one change, invoke os-lab-scorer on <path/to/target-skill>, act on KEEP or DISCARD. If no baseline exists, have os-lab-scorer record it first.

Workflow

1. Orient (Stage 0): read context/memory/improvement-ledger.md, the target's evals/results.tsv, references/program.md, the test registry, context/memory.md, recent surveys, the last 100 lines of context/events.jsonl, and open map debt (Repeat: YES first). Write the hypothesis and register the cycle. See stage-0-orientation.md.
2. Mutate (Stage 1): classify the failure, propose one change, log friction events as they occur. See mutation-protocol.md and stage-1-execution.md.
3. Score (Stage 2): invoke os-lab-scorer. With a PEER_AGENT, the peer scores independently from its own session. See stage-2-verification.md.
4. Decide (Stage 3): KEEP is applied and committed; DISCARD writes a correction packet and re-signals the inner agent. See stage-3-decision.md.
5. Repeat from step 2 until the cap, the target score or the budget guard.
6. Close (Stage 4): friction gate, surveys, post-run metrics, session log, loop report, test registry, improvement ledger, then os-lab-memory for L3 promotion, auto-memory, Triple-Loop trigger check and lock release. See stage-4-close.md.