agent-token-optimization-patterns

Agent Token Optimization Patterns

Overview

Token = cost + latency. Every token you can save without losing quality is pure value — lower bills, faster responses, and more headroom before context limits bite. At scale, unoptimized agents burn 3-10× more tokens than necessary, primarily through context stuffing, wrong-model routing, and cache thrashing.

Token optimization is not about cutting corners. It is about precision: sending exactly the information the model needs, in exactly the right form, to the right model, at the right time.

Quick Reference

Technique	Token Savings	Implementation Complexity	Risk
Prompt caching (stable content first)	45-80% on cache hit	Low	Cache thrashing if content rotates
Model routing (haiku for simple tasks)	60-90% cost reduction	Medium	Quality degradation on misrouted tasks
Structured data over prose	20-40%	Low	Schema design overhead
Progressive summarization	30-60%	Medium	Lossy compression of earlier context
System prompt deduplication	10-30%	Low	Divergence if copies drift out of sync
Context slicing (role-based)	20-50%	Medium	Missing context if slices are too narrow
Sliding window (drop oldest turns)	Variable	Low	Loss of early conversation context
Spawn new agent vs stuff context	High (resets window)	High	Coordination overhead, handoff cost

Context Engineering

Context engineering is the systematic practice of controlling what goes into an agent's context window. The framework has four operations: Write, Select, Compress, Isolate.

Write — Craft Precise Prompts

Write prompts that express the task in minimum tokens without ambiguity. Prefer imperative verbs over explanatory prose. Replace "Could you please help me understand..." with "Explain:". Remove politeness markers, hedges, and meta-commentary — the model does not need them.

// WRONG: verbose and hedging
const verbose = `
  I was wondering if you could help me take a look at the following code
  and maybe identify any potential issues that might be present in it.
  Please be thorough but also concise in your response if possible.
`;

// CORRECT: imperative, direct
const precise = `Review this code. List issues by severity (critical/high/medium/low). One line each.`;