api-rate-limiting-throttling

API Rate Limiting and Throttling

Overview

Rate limiting protects services from traffic spikes, abuse, and accidental overload. Choosing the wrong algorithm leads to either boundary spikes that allow bursting through limits, or excessive rejection of legitimate traffic. Use this guide to implement, review, or debug rate limiting logic.

When to use: Designing public or internal APIs; reviewing middleware for throttling correctness; evaluating Redis-based distributed limiting; auditing rate limit response headers; checking client-side retry and backoff behavior.

Quick Reference

Algorithm	Burst Tolerance	Accuracy	Complexity	Best For
Token Bucket	High — refills at rate R, allows bursts up to capacity C	Good	Medium	APIs that allow short bursts
Leaky Bucket	None — constant drain rate	Good	Medium	Smoothing traffic to downstream
Sliding Window Counter	High — no boundary spikes	Excellent	Medium-High	Accurate per-user limits
Fixed Window Counter	Medium — full quota resets at boundary	Fair	Low	Simple counters, background jobs
Distributed (Redis Lua)	Configurable	Excellent	High	Multi-instance production APIs

---

Patterns in Detail

1. Token Bucket Algorithm

The token bucket holds up to capacity tokens. Tokens are added at refillRate per second. Each request consumes one token. Requests that arrive when the bucket is empty are rejected or queued.

Red Flags:

• Storing last-refill timestamp as an integer — truncation error accumulates over time
• Not capping tokens at capacity — bucket grows unboundedly after idle periods
• Per-process in-memory state in multi-instance deployments — each instance has a full bucket

TypeScript:

interface TokenBucket {
  tokens: number;
  lastRefillMs: number;
  readonly capacity: number;
  readonly refillRatePerMs: number;
}

function createTokenBucket(capacity: number, refillRatePerSecond: number): TokenBucket {
  return {
    tokens: capacity,
    lastRefillMs: Date.now(),
    capacity,
    refillRatePerMs: refillRatePerSecond / 1000,
  };
}

function consumeToken(bucket: TokenBucket): { allowed: boolean; bucket: TokenBucket } {
  const now = Date.now();
  const elapsed = now - bucket.lastRefillMs;
  const refilled = Math.min(
    bucket.capacity,
    bucket.tokens + elapsed * bucket.refillRatePerMs,
  );
  if (refilled < 1) {
    return { allowed: false, bucket: { ...bucket, tokens: refilled, lastRefillMs: now } };
  }
  return { allowed: true, bucket: { ...bucket, tokens: refilled - 1, lastRefillMs: now } };
}