autoresearch-core-skill

What I do

I am the methodology source for the autoresearch iteration protocol — a fully autonomous, metric-driven loop that modifies a system, runs a mechanical evaluator, and keeps or reverts the change based on a falsifiable verdict. I do not self-judge: every keep/revert decision comes from an evaluator that emits {"pass":bool,"score":N}. I treat the 5 stages below as a single indivisible unit; a "loop" that skips Evaluate or drops the keep/revert step is not autoresearch. Domain skills (autoresearch-ml-skill, -code-skill, -research-skill) and retrofitted skills cite my references/*.md by path rather than duplicating this body.

The 5-Stage Loop

Every iteration runs all five stages in order. Skipping a stage breaks the protocol.

1. Understand — Read the audit trail (last 10–20 TSV rows, git log --oneline -20, git diff HEAD~1 on the last kept commit). Decide what worked, what failed, what is untried.
2. Hypothesize — Propose ONE falsifiable change ("if I do X, the metric will move in direction Y"). Atomic, single logical unit.
3. Experiment — Apply the change, commit it (experiment: <description>), run the evaluator.
4. Evaluate — The evaluator emits {"pass":bool,"score":N}. pass decides keep/revert; score is logged to the TSV. No LLM self-judgment in this decision.
5. Log & Iterate — Append the TSV row, regenerate progress.png if configured, then loop back to Understand.

Evaluator Contract

The evaluator is the only source of keep/revert truth. It must emit exactly this JSON shape on stdout's last line:

{"pass":bool,"score":N}

• pass: true → keep the commit; pass: false → revert (git reset --hard HEAD~1 or git revert HEAD --no-edit).
• score: N → numeric (int or float) logged to the results TSV. Direction (higher_is_better / lower_is_better) is set at init; the evaluator itself emits a raw number.
• Guard commands (e.g. npm test) run AFTER evaluate; a failed guard forces revert regardless of pass.