Dependency Management

Overview

Every external dependency is a liability. Every network call can fail, every downstream service can become slow, every database can become overloaded. Dependency management is the discipline of treating every external interaction as unreliable and building defenses that keep your service healthy even when dependencies are not.

At Amazon, a single service may depend on dozens of other services, databases, caches, and external APIs. Without proper dependency management, a slowdown in one dependency cascades into a full system outage—the "thundering herd" or "retry storm" that has caused some of Amazon's most significant incidents.

The core principle: your service's availability is the product of its dependencies' availabilities, unless you actively decouple from them. If you depend on 5 services each at 99.9% availability and do nothing, your service can only be 99.5% available. Proper dependency management breaks this multiplication.

When to Use

• Making any HTTP call to another service
• Querying any database or cache
• Calling any third-party API
• Publishing to or consuming from message queues
• Reading from configuration services
• Any I/O operation that crosses a network boundary

Amazon Context

Amazon's distributed architecture means most requests traverse multiple services. The call graph for a single customer request might involve 50+ services. Without dependency isolation, a single slow service can cause a cascading failure that takes down the entire customer experience.

Key lessons learned:

• The 2004 outage taught Amazon that retries without backoff amplify failures (retry storms)
• The "thundering herd" incidents taught that all callers recovering simultaneously overwhelms the recovering service
• The "gray failures" (slow but not dead services) are harder to detect and more damaging than hard failures

Amazon's architecture mandates: circuit breakers at every service boundary, timeouts on every call, retries with jitter, and bulkheads isolating failure domains. These are not optional patterns—they are architectural requirements enforced through ORR.