Cache Stampede: The Four Patterns That Actually Work

Stampede: a hot key expires, a thousand requests all miss at once, they all hit the database, the database falls over, the cache fills up again — until it expires. This is a recurring, very expensive outage pattern. Pick one of the four patterns below before it happens.

1. Mutex (single-flight)

On a miss, acquire a short-lived lock on a sibling key and have exactly one request recompute. Others either wait or serve a stale value.

SET lock:user:42 myuuid NX PX 3000
# if OK -> I am the recomputer; others retry in 50ms

2. Probabilistic early expiration (XFetch)

Each request checks the remaining TTL. If TTL is short relative to the compute cost, refresh probabilistically. Over many requests, exactly one of them will refresh early and everyone else still hits the cache.

const shouldRefresh =
  now - deltaEstimate * Math.log(Math.random()) >= expiresAt;

This is the Vatto/Aulbach "XFetch" algorithm. It's stateless and doesn't need a lock.

3. Stale-while-revalidate

Store two TTLs: a soft TTL (serve stale past this point) and a hard TTL (actually evict). On a soft miss, serve the stale value and kick off a background refresh.

4. Pre-warmed hot keys

For a handful of known-hot keys, don't let them expire at all. A cron job overwrites them on a schedule. You give up eventual-consistency nuance for zero-miss latency — a reasonable trade for the top 50 keys.

Rule of thumb

• Expensive compute, many readers → mutex.
• Cheap compute, very many readers → XFetch.
• You can tolerate stale briefly → stale-while-revalidate.
• You know the hot set in advance → pre-warm.

Combine patterns — they're not exclusive. The worst option is none.