rate-limiting

Add rate limiting

A rate limit is a promise about fairness and safety: no single caller can consume the service, and callers are told how to behave. Decide what you limit (a key), how (an algorithm), where the counters live (shared, atomic), and what clients see; then test the boundary exactly, including under concurrent requests. Sources: RFC 6585 (429), RFC 9110 (Retry-After), IETF draft-ietf-httpapi-ratelimit-headers (RateLimit, RateLimit-Policy), OWASP Authentication and Credential Stuffing Prevention cheat sheets, Redis documentation on atomic scripts.

Steps

1. Name the threat and the key. Credential stuffing, a noisy client, scraping, or a downstream to protect. The key follows: user id, API key, tenant, IP (weakest: NAT and IPv6 rotation; use a /64 for IPv6), or a combination. Derive the IP from the trusted proxy chain only, never a raw client-supplied X-Forwarded-For. Call search_lore for existing limiter middleware.
2. Choose the algorithm for the behaviour you want. Token bucket / GCRA: bursts up to capacity, steady refill (most APIs). Sliding-window counter: smooth rate, little memory. Fixed window: simplest, allows 2× bursts at window edges. Concurrency limit (semaphore): protects a downstream's capacity, not a rate. Use the repo's, gateway's or a maintained library's implementation before writing one.
3. Share and atomically update the counters. An in-process counter behind a load balancer is not a limit. Use the shared store with one atomic operation per decision: a Redis Lua script or INCR whose returned value is compared, never GET then SET (concurrent requests all read the same value and all pass). Set the TTL in the same atomic step: an INCR followed by a separate EXPIRE that fails leaves a key that never expires, locking the caller out forever. Use the store's clock (Redis TIME) so instance clock skew cannot widen the window.
4. Fail open or closed on purpose. If the store is unavailable: fail open for ordinary reads (availability), fail closed or fall back to a conservative local limit for login, password reset, OTP and costly writes. Put a short timeout on the store call. Comment the decision and test it.
5. Tell the client. Reject with 429 and Retry-After (seconds). Emit the repo's rate-limit headers; if the repo has none, use the current IETF draft fields, RateLimit-Policy: "default";q=100;w=60 and RateLimit: "default";r=42;t=18, on successful responses as well as rejections. Retry-After takes precedence over RateLimit. Legacy X-RateLimit-* headers stay if clients depend on them. Never leak other tenants' identifiers in a partition key. Document it (the openapi-contract skill).
6. Throttle authentication without enabling lockout abuse. Limit per account and per source; prefer progressive delays or a challenge over hard account lockout (an attacker can lock a victim out); return the same response whether or not the account exists. Never disable a security limit for convenience.
7. Check early and cheaply. Run the limiter before expensive work (body parsing, downstream calls) where the key is available; for authenticated keys, after cheap token validation. Keep key derivation allocation-free.
8. Leave a way out. Allow-lists for internal callers, per-tenant overrides in config, and a rejection metric by limit name (not by user id; cardinality) with an alert, so a limit locking out a real customer is seen within minutes.