prompt-caching

Prompt caching (Claude API)

Claude's prompt caching reduces repeated-prefix cost by up to 90% and can cut latency significantly. It also has a 5-minute TTL on the default cache tier — if you're not hitting within 5 minutes, the cache entry is gone. Design for this.

When caching helps

Caching helps when:

• You make many requests with overlapping prefixes (system prompt, tools definitions, long documents).
• Requests are frequent enough to hit within the TTL.
• The cacheable content is ≥1024 tokens (Sonnet/Opus) or ≥2048 (Haiku) — smaller caches don't pay off.

Caching doesn't help when:

• Each request is one-shot with a totally unique prompt.
• The prompt content varies above the cache boundary.
• Traffic is too sparse to hit the TTL window.

How it works (at a high level)

You mark the end of cacheable content with a cache_control breakpoint:

messages = [
  {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "<long document or system instructions>",
        "cache_control": {"type": "ephemeral"}
      },
      {
        "type": "text",
        "text": "<the per-request query>"
      }
    ]
  }
]

Everything up to and including the marked block is cached. Anything after is the variable part.

On a cache hit, you pay a reduced rate for the cached tokens and normal rate for new tokens. On a miss, the first request incurs a cache-write cost (higher than normal), and subsequent requests within 5 minutes hit.

Design rules

1. Order content by stability

Put the most stable content first, least stable last.

Good:

[stable system prompt]
[stable tool definitions]
[stable shared context]
<cache breakpoint>
[variable per-request query]

Bad:

[variable per-request query]
[stable system prompt]
<cache breakpoint>

The variable part at the top breaks the cache on every call.

2. Use up to 4 breakpoints

Claude supports multiple cache breakpoints. Use them to cache at multiple layers: