ai-llm

LLM Engineering - Lifecycle Skill

Modern Best Practices: treat the model as a versioned component with contracts, eval gates, rollout controls, cost budgets, and explicit fallback paths. Prefer stable decision criteria over static "best model" lists, and verify volatile provider facts against current official docs before recommending a stack.

This skill is the umbrella skill for deciding how to build, adapt, evaluate, migrate, and operate LLM systems.

Use this skill for architecture and lifecycle decisions. Use sibling skills for implementation depth.

No theory. No generic AI history. Focus on operational choices, tradeoffs, checklists, and reusable templates.

ASCII Flow

LLM product need
  |
  v
outcome contract
  task + users + quality + latency + cost + privacy/compliance
  |
  v
architecture choice
  prompt-only -> RAG -> tools/agents -> adaptation/fine-tuning -> hybrid
  |
  v
evaluation and rollout
  golden set + edge cases + canary + observability + rollback
  |
  v
operated model component
  versioned prompts/models/configs + fallbacks + governance

When to Use This Skill

Activate this skill when the user asks for:

• Choosing between prompt-only, RAG, tool use, fine-tuning, or hybrid LLM architectures
• Selecting a provider, model tier, or deployment path for a production workload
• Planning or reviewing model/provider migrations
• Designing eval suites, graders, rollout gates, and regression policies
• Calculating cost, latency, and quality tradeoffs at the system level
• Defining LLM governance: data handling, safety boundaries, compliance, and rollback
• Building current recommendations that depend on provider/platform capabilities
• Creating a preflight plan for an LLM project before implementation begins

Scope Boundaries (Use These Skills for Depth)

• Choosing the approach first (classical ML vs LLM vs RAG vs fine-tune vs agent) -> ai-architecture-advisor
• Eval methodology: LLM-judge bias, framework choice, threshold calibration, reproducibility -> ai-evals
• Prompt design, structured outputs, prompt CI/CD -> ai-prompt-engineering
• RAG design, chunking, retrieval, reranking -> ai-rag
• Agent architectures, MCP tools, multi-agent orchestration -> ai-agents
• Serving optimization, batching, routing, quantization -> ai-llm-inference
• Deployment, monitoring, incident response, security depth -> ai-mlops
• Hugging Face-specific LLM training workflows (TRL, SFT, DPO, GRPO) -> use the huggingface-skills: plugin (external)
• Build a transformer/GPT and BPE tokenizer from scratch (pre-training layer) -> ai-pretraining
• Multi-GPU pre-training: FSDP, DeepSpeed ZeRO, tensor/pipeline parallelism, reproduce GPT-2 -> ai-distributed-training
• Scaling laws, Chinchilla compute-optimal sizing, token/param budget -> ai-scaling-laws
• Web-scale + synthetic pre-training corpus curation and data ablations -> ai-data-curation-pretraining
• Post-training depth: RLHF/PPO, DPO, GRPO, RLVR, reward modeling, alignment, and reasoning-model training -> ai-post-training