evaluator-optimizer

Evaluator-Optimizer

Generate → evaluate against explicit criteria → refine with the feedback → repeat until it passes or a round cap is hit. Use it when two things are both true: there's a clear bar (tests, a lint rule, a stated requirement — not "make it better") and refinement demonstrably helps (a model can act on concrete feedback better than it produced the first draft blind). If either is missing — no checkable criteria, or one attempt is already as good as five — skip the loop; it just burns rounds for no gain.

This is a different shape from advisor-checkpoint: that skill is one strategic consult before you commit to an approach. This is a loop that converges one artifact against a bar you can actually check.

Prefer an objective check over an opinion

Whenever the task has one, run the real check — a test suite, go vet/swiftlint/a build — rather than asking a model to judge. A test result is ground truth; a model's opinion about whether code "looks correct" is not. Reserve a model-as-evaluator for criteria that genuinely can't be mechanically checked (architecture quality, whether a document actually answers the stated question, prose clarity).

The loop

1. Generate. Produce a candidate against the task.
2. Evaluate. Run the objective check, or — if there isn't one — dispatch an evaluation only against explicit criteria you state up front (not "is this good," but "does it satisfy: correctness, no new lint violations, handles the empty-input case"). The evaluator's job is to grade, not to fix — keep the roles separated so the feedback is a clean signal, not a silent rewrite.
3. Refine. If it fails, feed the concrete failure (the test output, the lint error, the evaluator's specific complaint) back into the next generation — not "try again," but "this failed because X."
4. Repeat, capped at 3-5 rounds. If it hasn't converged by then, stop and surface the failure rather than keep spinning — a persistent failure after several rounds usually means the criteria are wrong, the task is underspecified, or the approach needs to change, not that round 6 will suddenly pass.