prompt-tuning

Prompt tuning — working with non-deterministic output

Every rule here has a measured cost behind it. The first one alone accounts for three tuning rounds spent moving a number that had never moved.

Measure before you change anything

1. Take a noise baseline BEFORE the first edit. Same prompt, same inputs, at least two runs; record the spread of every metric you care about. A delta smaller than the spread is noise, not the effect of your change. To compare two versions, run each at least three times per input and compare medians.

Measured: ±20% output length and a jumping paragraph count with no change to the prompt at all. Three rounds of tuning were spent chasing that before anyone ran the same prompt twice.

2. Measure conversion factors between representations; never assume them. One initiative assumed a 1.15× expansion between two languages and measured 1.3–1.4×. The assumption would have invalidated the entire budget built on it. When a factor's spread is wider than your target band, no source-side budget can guarantee the target — catch the tail with a gate after the transformation instead of tightening the budget before it.

3. After three failed rewordings of the same defect on the same case, name it a KNOWN LIMITATION and hand it to a human. A fourth blind iteration is chasing noise with extra steps.

Writing the prompt

4. Every numeric example becomes a quota; every template phrase becomes a verbatim anchor. Describe the CRITERION, not the illustration. "Two threads means two paragraphs" produced two-paragraph output in six samples out of six, where an example of a two-paragraph answer had not. State criteria in both directions — a one-sided formulation works only in the direction you named.

5. A rule in a prompt is a request. Enforcement is a mechanism that runs AFTER generation — a gate plus regeneration with a targeted nudge. Confirmed on four separate properties: length ceiling, presence of figures, language of a recommendation, and post-translation condensation.