synthetic-data-finetuning-expert

Synthetic Data & Fine-Tuning Expert (Custom Domain SLMs)

English | Bahasa Indonesia

---

English

Orchestration & Integration

Connects and orchestrates with domain skills like local-slm-edge-ai-expert, ai-prompt-engineering-expert, ai-evals-benchmark-expert, python-programming-expert, and ai-cost-token-optimizer to build high-performance, cost-effective domain models.

Description

Production-grade guide for generating synthetic training datasets, curating high-signal instruction pairs, executing Parameter-Efficient Fine-Tuning (QLoRA / LoRA) with Unsloth and Hugging Face TRL, performing Direct Preference Optimization (DPO), and quantizing custom Small Language Models (SLMs) to GGUF for edge or on-premise execution.

Swarm Synergy: Within the AI Engineering Swarm, this skill serves as the Model Specialization Lead. When frontier API costs or latency become prohibitive, it trains, aligns, and deploys hyper-efficient domain SLMs (1B–8B parameters) in Phase 4.

Trigger Conditions

• Generating domain-specific synthetic training data from seed documents, codebases, or APIs.
• Filtering low-quality or hallucinated synthetic data using LLM-as-a-judge curation pipelines.
• Fine-tuning open-weights models (Llama 3.3, Qwen 2.5, Mistral) on custom tasks using 4-bit QLoRA.
• Aligning model outputs using Direct Preference Optimization (DPO) to enforce specific response styles.
• Quantizing fine-tuned models to GGUF (q4_k_m, q8_0) for zero-latency local inference with Ollama or llama.cpp.

Synthetic Data & Fine-Tuning Lifecycle

1. SEED EXTRACTION & SYNTHESIS
   [Raw Docs / Codebase] ──► [Frontier LLM / Distilabel] ──► Raw Instruction Pairs (10k+)

2. QUALITY FILTERING (LLM-AS-A-JUDGE)
   Raw Instruction Pairs ──► [Rubric Scorer / De-duplication] ──► Curated Gold Dataset (2k-5k)

3. 4-BIT QLORA FINE-TUNING (UNSLOTH)
   Base Model (e.g. Qwen 2.5-Coder) + LoRA Adapters ──► SFT / DPO Training Loop

4. QUANTIZATION & LOCAL DEPLOYMENT
   Merged 16-bit Weights ──► [llama.cpp GGUF Export] ──► Local Ollama Service (<50ms latency)