minicpm5-finetune-gguf-lora

Fine-tune MiniCPM5-1B → GGUF LoRA adapter

The framework skills (minicpm5-finetune-*) all emit a PEFT adapter (adapter_model.safetensors + adapter_config.json). But llama.cpp / llama-server — and the MiniCPM Desk Pet app's custom-LoRA upload — load a GGUF LoRA adapter (--lora some-adapter.gguf). This skill is the bridge: train a PEFT LoRA, then convert it to GGUF and run/upload it.

This is the path you want when the base model is served as GGUF (Desk Pet, Ollama, LM Studio, plain llama-server). If you serve the fp16 HF base with vLLM / transformers, you don't need GGUF — load the PEFT adapter directly.

Pipeline overview

  train (any minicpm5-finetune-* skill)        this skill
 ┌────────────────────────────────────┐   ┌──────────────────────────────────┐
 BASE (fp16 HF) ─► adapter_model.safetensors ─► convert_lora_to_gguf.py ─► adapter.gguf
                   adapter_config.json                                          │
                                                                  llama-server --lora / Desk Pet upload

Required input

Var	Example	Default
ADAPTER_DIR	./runs/minicpm5_unsloth/adapter_final	required — a PEFT dir with adapter_config.json + adapter_model.safetensors
BASE_MODEL	openbmb/MiniCPM5-1B (HF id) or a local fp16 HF dir	required — must be the same base the adapter was trained on
OUTTYPE	f16 (recommended) / q8_0 / bf16 / f32	f16
OUT_GGUF	./minicpm5-mylora.gguf	<ADAPTER_DIR>/adapter_model.f16.gguf

Don't have an adapter yet? First run a training skill — start from the router minicpm5-finetune (or go straight to minicpm5-finetune-unsloth for single-GPU LoRA). Come back here with ADAPTER_DIR pointing at its output.

Steps

1. Get llama.cpp (has the converter)

git clone --depth=1 https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
pip install -r requirements.txt          # converter deps: torch, safetensors, gguf, transformers