building-with-llms

Building with LLMs

Scope

Covers

• Building and shipping LLM-powered features/apps (assistant, copilot, light agent workflows)
• Prompt + tool contract design (instructions, schemas, examples, guardrails)
• Data quality + evaluation (test sets, rubrics, red teaming, iteration loop)
• Production readiness (latency/cost budgets, logging, fallbacks, safety/security checks)
• Using coding agents (Codex/Claude Code) to accelerate engineering safely

When to use

• “Turn this LLM feature idea into a build plan with prompts, evals, and launch checks.”
• “We need a system prompt + tool definitions + output schema for our LLM workflow.”
• “Our LLM is flaky—design an eval plan and iteration loop to stabilize quality.”
• “Design a RAG/tool-using agent approach with safety and monitoring.”
• “We want to use an AI coding agent to implement this—set constraints and review gates.”

When NOT to use

• You need product/portfolio strategy and positioning (use ai-product-strategy).
• You need a full PRD/spec set for cross-functional alignment (use writing-prds / writing-specs-designs).
• You need primary user research (use conducting-user-interviews / usability-testing).
• You are doing model training/research, infra architecture, or bespoke model tuning (delegate to ML/eng; this skill assumes API models).
• You only want “which model/provider should we pick?” (treat as an input; if it dominates, do a separate evaluation doc).
• You want to design an eval/benchmark framework without building a specific feature (use ai-evals).
• You need to evaluate a vendor/tool for adoption rather than build an LLM feature (use evaluating-new-technology).
• You want to quickly prototype or vibe-code an idea without production planning (use vibe-coding).

Inputs

Minimum required

• Use case + target user + what “good” looks like (success metrics + failure modes)
• The LLM’s job: generate text, transform data, classify, extract, plan, or take actions via tools
• Constraints: privacy/compliance, data sensitivity, latency, cost, reliability, supported regions
• Integration surface: UI/workflow, downstream systems/APIs/tools, and any required output schema