prompt-engineer

Prompt Engineer

Overview

This skill turns vague intent into a precise, testable prompt and provides a repeatable loop for measuring and improving prompt quality. It covers prompt anatomy, role/system framing, few-shot example selection, output contracts (JSON/schema), reasoning scaffolds (chain-of-thought, decomposition), and rigorous evaluation (golden sets, rubrics, LLM-as-judge, pairwise comparison).

Keywords: prompt engineering, system prompt, few-shot, chain-of-thought, output format, JSON mode, structured output, hallucination, refusal, LLM-as-judge, eval rubric, prompt optimization, A/B prompt, temperature, prompt template.

Use this skill whenever the user is authoring, debugging, or comparing prompts — not when they need a finished answer to the underlying question itself.

When to use vs. not

• USE: "Write/improve a prompt", "make output reliably JSON", "model ignores instructions", "build an eval for my prompt", "pick few-shot examples", "reduce hallucinations".
• DON'T USE: the user just wants the task done once (answer the question directly). Only invoke prompt engineering when the prompt artifact is the deliverable or the problem.

Process

Follow these steps in order. Skip a step only with a stated reason.

1. Clarify the job. Pin down: the task, the consumer of the output (human vs. parser/code), success criteria, hard constraints (length, format, forbidden content), and failure modes seen so far. If two or more of these are unknown and the prompt is non-trivial, ask before writing.
2. Choose the prompt pattern. Match the task to a pattern using the table in references/patterns.md (extraction, classification, generation, transformation, agentic/tool-use, judge). The pattern dictates structure and which scaffolds matter.
3. Draft with the anatomy. Assemble the prompt from the canonical sections below. Put durable instructions in the system prompt; put per-request data in the user message.
4. Add an output contract. If a machine reads the output, specify an exact schema, give a filled example, and instruct "output only the JSON, no prose". See references/output-contracts.md.
5. Add reasoning scaffold only if needed. Use step-by-step / decomposition for multi-step reasoning. Keep reasoning OUT of structured output (use a separate field or a scratchpad you discard).
6. Select few-shot examples (2-5) that cover edge cases and the exact target format. Order matters; cover the hard cases. See references/few-shot.md.
7. Set decoding params. Recommend temperature/top-p for the task (deterministic extraction → low temp; creative → higher). See the params table in references/patterns.md.
8. Build an eval. Create a small golden set (10-50 cases) and a scoring method (exact match, rubric, or pairwise LLM-judge). Use scripts/eval_prompts.py to run variants and compute pass rates. See references/evaluation.md.
9. Iterate. Change ONE variable at a time, re-run the eval, keep what wins. Record results in the table from templates/prompt-spec.md.