redis-caching-and-pubsub

Redis Caching & Pub/Sub Architecture Guide

Production-grade guide for designing, implementing, and scaling Redis as a high-performance cache, session store, real-time pub/sub message broker, and distributed coordinator.

---

1. Caching Strategies & Design Patterns

1.1 Cache-Aside (Lazy Loading)

The application reads from the cache first. On a cache miss, it reads from the primary database, updates the cache, and returns the result.

+-------------+      1. Read Key      +-------+
| Application | --------------------> | Cache |
|             | <-------------------- |       |
+-------------+     2. Cache Hit      +-------+
       |
       | 3. Cache Miss
       v
+---------------+
| Primary DB    |
+---------------+

• Best For: Read-heavy workloads with unpredictable access patterns.
• Trade-off: Stale data risk if DB writes don't invalidate/update cache; higher latency on cache miss.

1.2 Write-Through Caching

The application writes data directly to the cache, and the cache synchronously writes to the database.

• Best For: Mission-critical reads where cache staleness cannot be tolerated.
• Trade-off: Higher write latency; potential cache pollution with infrequently read data.

1.3 Write-Behind (Write-Back) Caching

The application writes to the cache immediately, and an asynchronous process flushes cache updates to the underlying DB in batches.

• Best For: Write-heavy systems (e.g., analytics, metrics aggregators, counter updates).
• Trade-off: Risk of data loss if Redis crashes before flushing to primary DB.

1.4 Cache Stampede (Thundering Herd) Mitigation

When a popular cache key expires, thousands of concurrent requests hit the backend database simultaneously.

Solution A: Distributed Mutex / Lock

Acquire a lock to recompute the cache value; force other concurrent requests to wait or return stale data.

Solution B: Probabilistic Early Expiration (XFetch Algorithm)

As a key nears expiration, probabilistically refresh it in the background before it expires.