model-selection

Which model, and why

The wrong question is "how important is this task?" — everything feels important. The right question is about the shape of the work.

The decisive test: is there a cheap verifier downstream?

If a mistake in this output would be caught mechanically — by a compiler, a test, a schema, a deterministic checker, a later verification pass — a cheaper model is usually correct. The verifier is the quality control; the generator only has to get close. Wasting a frontier model on work that is checked for free buys nothing.

If this output IS the last line of defence — nothing downstream can tell whether it is wrong — spend the strong model. When the model is the verifier, its judgement is the product.

Everything below follows from that one test.

Spend the most capable model on

• Novel design with interacting constraints. Several forces pulling against each other (a transaction boundary against an external store, an authorization model against an inheritance hierarchy), where the failure is a subtle wrong trade-off rather than a broken build.
• Adversarial verification where a false negative is expensive. Data-loss paths, authorization, money, migrations. The job is to refute a plausible claim — precision work, and nothing catches a wrong "this is fine".
• Root-causing a defect that survived cheaper attempts. If two passes have already failed, the problem is not effort, it is depth. Escalate rather than retry.
• Reasoning about validity. Experiment design, threats to validity, whether an inference actually follows from the evidence. This is the work most damaged by a confident, shallow answer.
• Consequences invisible at the edit site. "What else does this change break?" is exactly the class cheap models miss and the class that costs most.
• Anything that will be published or acted on without further checking — an architecture decision, a claim in an article, a security conclusion.

Use a cheaper, faster model for