gemma-trainer

Gemma Training and Fine-Tuning Skill

1. Core Principles: Local Fine-Tuning Setup

When training locally, memory efficiency and execution speed are huge. Always guide the user to follow these best practices:

• Prioritize Unsloth: For local single-GPU training, always recommend Unsloth. It supports Gemma 4 natively, uses up to 70% less memory, and is up to 2x faster than standard Hugging Face PEFT training.
• Fall Back to TRL: For multi-GPU environments (using DDP/FSDP) or when Unsloth is unavailable, use Hugging Face TRL (SFTTrainer, DPOTrainer) coupled with PEFT and bitsandbytes (for QLoRA).
• Always use QLoRA (4-bit Quantization): Crucial for fitting Gemma models (like Gemma 4 12B/31B) into consumer VRAM.
• Manage Context Window & Max Length: Although Gemma 4 supports up to a 256K context window, recommend training with a context window of 2048 to 8192 tokens locally to prevent Out-Of-Memory (OOM) errors.

2. Choosing the Right Training Method

Help the user choose the correct workflow based on their goal:

• Supervised Fine-Tuning (SFT): Teaching new domains, specialized task instructions, or custom output structures.
  • Prerequisites: Raw text, instruction pairs, or chat logs.
  • Output: Adaptor trained on prompt/completion pairs.
• Direct Preference Optimization (DPO): Aligning model style, behavior, tone, or safety with human preferences.
  • Prerequisites: A previously SFT-trained Gemma model and preferred pairwise datasets.
  • Output: Aligning model weights directly without a separate reward head.
• Reward Modeling (RM): Training a scoring system to evaluate response quality.
  • Prerequisites: Binary preference pairwise datasets.
  • Output: A classification-style reward head on top of Gemma.

3. Dataset Preparation & Validation

Formatting issues are the #1 cause of poor training runs. Ensure you validate files using the utility script [assets/dataset_prep.py].