peft

PEFT: Parameter-Efficient Fine-Tuning

Overview

PEFT (Parameter-Efficient Fine-Tuning) provides methods like LoRA, QLoRA, and adapters for efficiently fine-tuning large language models by training only a small fraction of parameters. Apply this skill for memory-efficient fine-tuning, LoRA configuration, adapter training, and producing lightweight model checkpoints.

When to Use This Skill

This skill should be used when:

• Fine-tuning large models limited GPU memory on- Implementing LoRA (Low-Rank Adaptation)
• Using QLoRA for 4-bit/8-bit fine-tuning
• Adding adapters to transformer models
• Applying prefix tuning or prompt tuning
• Training only small percentage of model parameters
• Creating efficient model checkpoints
• Domain adaptation for LLMs

Quick Start

Basic Import and Setup

from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

LoRA Fine-tuning

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    "gpt2",
    load_in_8bit=True,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("gpt2")

# Configure LoRA
lora_config = LoraConfig(
    r=16,  # LoRA rank
    lora_alpha=32,  # LoRA scaling parameter
    target_modules=["c_attn", "c_proj"],  # Modules to apply LoRA
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM
)

# Apply LoRA to model
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Output: trainable params: 786,432 || all params: 124,646,080 || trainable%: 0.63

Training

from peft import LoraConfig, get_peft_model, TaskType
from transformers import TrainingArguments, Trainer
from datasets import load_dataset

# Prepare dataset
dataset = load_dataset("imdb", split="train[:1000]")

def tokenize_function(examples):
    return tokenizer(examples["text"], truncation=True, max_length=512)

dataset = dataset.map(tokenize_function, batched=True)

# Training arguments
training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    learning_rate=3e-4,
    logging_steps=10,
    save_strategy="epoch",
)

# Create trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
)

# Train
trainer.train()