llm-finetuning

LLM Fine-Tuning (PEFT / LoRA)

Purpose

Fine-tuning changes model behaviour (format, style, task skill, domain language) from examples. It is the wrong tool for injecting frequently changing facts; use retrieval for that. Parameter-efficient methods such as LoRA train small low-rank adapters on frozen base weights, making single-GPU training of 7-8B models routine and letting one base model serve many adapters.

Agent Protocol

Trigger

"fine-tune", "LoRA", "QLoRA", "PEFT", "SFT", "instruction tuning", "DPO", "preference tuning", "adapter", "merge_and_unload".

Input Context

• Goal and why prompting/RAG is insufficient (with failing examples).
• Dataset: size, source, license, format, PII status.
• Base model and license; GPU type/VRAM; serving target.
• Eval set and success metric.

Output Artifact

Go/no-go decision, dataset spec, training config, eval plan, export/serving instructions.

Response Format

Decision: {fine-tune|RAG|prompting} because {reason}
Base model: {id} (license {x})  Method: {LoRA|QLoRA|full}  Stage: {SFT|DPO}
Data: {n} examples, chat format, split {train/val/test}
LoRA: r={r} alpha={a} dropout={p} targets={modules}
Train: lr={lr} epochs={e} eff_batch={b} max_len={L}
Eval: base vs tuned on {metric}; regression suite {name}
Export: {adapter|merged} -> {engine}

Completion Criteria

• Baseline (prompted base model, RAG) measured first.
• Held-out test set never used for tuning decisions.
• Chat template of the base model applied consistently in training and inference.
• Tuned model beats baseline on the target metric without regressions on general checks.

Should You Fine-Tune?

Goal	Best tool
New/changing facts, citations	RAG
Output format, tone, strict schema	Prompting + structured outputs first; SFT if still unreliable
Narrow task at lower cost/latency	SFT a smaller model (often distilled from a larger one)
Preference (helpfulness, style ranking)	DPO/ORPO after SFT
Domain vocabulary/style	SFT (or continued pre-training for large corpora)