loom-rate-limiting

Rate Limiting

Overview

Control the request rate a client can make: protect from abuse, enforce fair usage, shed load. Two decisions dominate correctness: which algorithm (burst tolerance vs accuracy vs memory) and how to make the counter atomic in a distributed setting. Everything else is headers and policy.

Algorithm Selection

Algorithm	Burst behavior	Accuracy	Memory/key	Use when
Fixed window	Allows 2× limit at window boundary	Poor	1 counter	Cheap, coarse limits where boundary burst is acceptable
Sliding window log	Exact, no boundary burst	Exact	O(limit) timestamps	Low limits needing precision (e.g. 5 login attempts)
Sliding window counter	Smooths boundary, small over/under	~99%	2 counters	General-purpose distributed limiting (best default)
Token bucket	Allows configurable burst up to capacity	Rate-exact avg	2 numbers (tokens, ts)	APIs that should tolerate bursts (most public APIs)
Leaky bucket	No burst; smooths to constant output	Shapes traffic	Queue	Protecting a fragile downstream at fixed throughput
GCRA	Burst = capacity, single value	Exact	1 timestamp (TAT)	High-throughput distributed limiting; token-bucket equivalent, cheaper

⚠ Fixed-window boundary burst is the classic footgun: with limit=100/min, a client can send 100 at 00:59.9 and 100 at 01:00.1 — 200 requests in ~0.2s while never violating either window. If bursts matter, use sliding-window or token-bucket.

⚠ Token bucket ≈ leaky bucket (as a meter) ≈ GCRA — mathematically equivalent rate meters differing in burst allowance and storage. Don't reimplement all three; pick token bucket for app code, GCRA for a single-value distributed limiter.

Token Bucket

Allows bursts up to capacity, refills at refillRate tokens/sec. Refill is computed lazily on access (no background timer needed).

class TokenBucket {
  private tokens: number;
  private lastRefill = Date.now();
  constructor(private capacity: number, private refillRate: number) {
    this.tokens = capacity;
  }
  consume(n = 1): boolean {
    const now = Date.now();
    this.tokens = Math.min(this.capacity, this.tokens + ((now - this.lastRefill) / 1000) * this.refillRate);
    this.lastRefill = now;
    if (this.tokens >= n) { this.tokens -= n; return true; }
    return false;
  }
}
// 100 req/min sustained, burst of 10:
const bucket = new TokenBucket(10, 100 / 60);