agent-cost-and-latency-spike-investigation

Agent Cost and Latency Spike Investigation

Purpose

A sudden cost or latency spike in one agent workflow is an incident, not an optimization project — the goal in the first hours is to scope it, identify what changed, and stop the bleeding, not to redesign the pipeline. llm-cost-and-latency-optimization covers the deliberate, scheduled work of reducing baseline cost/latency across an agent (right-sizing models, caching, batching); this skill covers the narrower, time-pressured question that comes first: why did this one workflow suddenly get more expensive or slower than it was yesterday, and what's the fastest safe action to take. Confusing the two wastes the window where a quick rollback would have worked and instead launches a multi-day optimization effort under incident pressure.

When to use

• A cost dashboard or billing alert shows a step-change increase for one agent/workflow/task type, not a gradual trend.
• p50/p95 latency for one specific workflow doubles (or worse) compared to its recent baseline, while other workflows are unaffected.
• An unexpected line item appears on an LLM provider invoice tied to a specific agent.
• Before scheduling a full cost/latency optimization pass — this investigation determines whether there's an active regression to fix first, so the optimization pass starts from a correct baseline.
• Deciding whether a spike is a regression (something broke) or legitimate growth (more users, more traffic) — these require entirely different responses.

Prerequisites & environment

• Per-call token usage and latency logging, segmented by workflow/task type (not just a single aggregate metric) — if the spike can't be isolated to one workflow because everything rolls into one dashboard number, segmenting the metrics is itself the first prerequisite to fix.
• A deploy/change log with timestamps: prompt edits, tool schema changes, model version or provider changes, retrieval index re-indexing runs, and infrastructure/routing changes — the single most useful artifact for this investigation is a timeline that can be laid next to the metrics timeline.
• Request volume metrics alongside cost/latency, so a spike in absolute cost can be distinguished from a spike in cost-per-request.
• Access to a recent-history transcript sample for the affected workflow (see agent-bad-response-triage-and-root-cause-classification for full-transcript capture practices) so the investigation isn't limited to aggregate numbers alone.