structured-output

Structured Output

Purpose

Standardize how an LLM app gets machine-readable output it can trust. Asking a model for JSON in the prompt and calling JSON.parse / json.loads on the result fails on a non-trivial fraction of calls, a stray markdown fence, a trailing comma, a hallucinated field, an unescaped quote, and those failures surface as production exceptions on a code path that "worked in testing." Provider-native structured output (schema-constrained decoding or strict tool schemas) drops the invalid-output rate by one to two orders of magnitude. This skill gives the decision order: use the strongest native mechanism the provider offers, validate every response against a schema, and wrap it in a bounded retry-with-error-feedback loop.

The reliability ladder (strongest first)

1. Schema-constrained output: the provider constrains decoding to your JSON Schema, guaranteeing the response validates. Anthropic output_config.format (json_schema), OpenAI response_format: json_schema with strict: true, Gemini/Vertex responseSchema. Use this whenever the model supports it.
2. Strict tool / function schema: when the value is really an argument to an action, route it through a tool with strict: true and additionalProperties: false. The provider validates the tool input against the schema.
3. Bare JSON mode: {"type": "json_object"} / "json" mode guarantees syntactically valid JSON but not your schema. Only acceptable with validation + retry on top.
4. Prompt-and-pray: "respond only with JSON" and hope. Never rely on this; it is the pattern this skill exists to replace.

Always add layers 1–3 with the validate-and-retry loop below, native constraints can still be interrupted by a length cap or a refusal.

The non-negotiable pair: validate, then retry with feedback

Even with native constraints, parse defensively:

1. Define the schema once (Zod / Pydantic / JSON Schema) and reuse it for both the request constraint and the response check.
2. Validate every response against it, never trust a raw parse.
3. On a validation failure, retry a bounded number of times, feeding the validator's error message back into the next request ("your previous output failed validation: ; return output matching the schema"). This self-correction recovers most transient failures.
4. Cap the attempts and surface a typed failure when exhausted, an unbounded correction loop is a cost and latency hazard.