scalability-playbook

Scalability Playbook

Systematic approach to identifying and resolving scalability bottlenecks.

Bottleneck Analysis

Current System Profile

Traffic: 1,000 req/min
Users: 10,000 active
Data: 100GB database
Response time: p95 = 500ms

Identified Bottlenecks

1. Database Queries

Symptom: Slow page loads (2-3s) Measurement: Query time p95 = 800ms Impact: HIGH - affects all reads Trigger: When p95 >500ms

2. Single Server

Symptom: High CPU (>80%) Measurement: Load average >4 Impact: MEDIUM - intermittent slowdowns Trigger: When CPU >70%

3. No Caching

Symptom: Repeated DB queries Measurement: Cache hit rate = 0% Impact: MEDIUM - unnecessary load Trigger: When query volume >10k/min

Scaling Strategies (Ordered)

Level 1: Quick Wins (Days)

1.1 Add Database Indexes

Problem: Slow queries Solution:

CREATE INDEX idx_users_email ON users(email);
CREATE INDEX idx_orders_user_created ON orders(user_id, created_at);

Expected Impact: 80% faster queries Cost: $0 Effort: 1 day

1.2 Enable Query Caching

Problem: Repeated queries Solution: Redis cache layer

const cached = await redis.get(`user:${userId}`);
if (cached) return JSON.parse(cached);

const user = await db.users.findById(userId);
await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));

Expected Impact: 60% reduction in DB load Cost: $50/month Effort: 2 days

Level 2: Horizontal Scaling (Weeks)

2.1 Add Read Replicas

Problem: Read-heavy workload Solution: Route reads to replicas

Write Load: Primary DB
Read Load: 3x Read Replicas

Expected Impact: 3x read capacity Cost: $300/month Effort: 1 week

2.2 Load Balancer + Multiple Servers

Problem: Single point of failure Solution:

ALB
 ├── Server 1
 ├── Server 2
 └── Server 3

Expected Impact: 3x throughput Cost: $400/month Effort: 1 week