load-balancing

Load Balancing

A load balancer's only job is to keep sending traffic to backends that can actually handle it, and stop sending it to ones that can't — everything else is a refinement of that. Most load-balancing incidents are not algorithm problems, they're health-check problems: a backend that's failing gets traffic anyway because the check never noticed, or a backend that's fine gets pulled because the check measured the wrong thing.

Correctness here matters more than cleverness. A load balancer that sends traffic to a dead backend has failed at its one job, no matter how good the algorithm is.

1. Choose L4 or L7 by what you need to inspect

L4 (transport-layer) balancing routes by IP and port without looking at the request — it's fast, protocol-agnostic, and can't make routing decisions based on content. L7 (application-layer) balancing terminates the connection and reads the actual request, so it can route by path or header, do TLS termination, and retry idempotent requests — at the cost of more compute and being protocol-specific (usually HTTP).

• Default to L4 for raw TCP/UDP workloads or when the balancer must stay protocol-agnostic.
• Use L7 when routing depends on the request itself — path-based routing, header-based canaries, or TLS termination all require it.
• L7 termination breaks true end-to-end TLS unless it re-encrypts to the backend — know which hop actually holds the certificate before assuming traffic is encrypted the whole way.

Done when: the balancing layer chosen matches whether routing decisions need request content.

2. Make health checks test the thing that actually matters

A health check that only confirms "the process is listening" will happily send traffic to a backend whose database connection died. The check should exercise a real dependency path — deep enough to catch actual failure, shallow enough not to become its own bottleneck or cascade a downstream outage into every backend failing its check simultaneously.