Skip to content
Prompt Words
History

Cache stampede

Also called: thundering herd, dogpile effect, request coalescing (the fix).

A popular cache key expires, and every request that arrives before it is filled again misses at the same moment. They all go to the database for the same row, so the database gets a sudden pile of identical reads. The fix lets one request reload the key while the others wait for its answer.

product:42 is a hot key in Redis. Expire it while 8 requests arrive, then turn on Coalesce misses and try again.

Requests: 0 · Cache misses: 0 · Database reads: 0

product:42 is in Redis, so requests for it never reach Postgres.

    Idle

    Say it in a prompt

    Protect the product:{id} cache in Redis from a stampede: on a miss, only one request per key reloads it from Postgres (single flight inside each server, plus a Redis lock SET lock:product:{id} NX PX 5000 across servers); the others wait up to 2 s for the new value, then read the cache again. Add ±10% jitter to the 300-second TTL.

    Vague vs precise prompt

    Vague prompt

    the database spikes every few minutes, fix it

    Typical resultAdds more database CPU or a bigger instance. The spikes get smaller but still come every 5 minutes, when the hot keys expire.

    Precise prompt

    Stop cache stampedes on product:{id}: on a miss let one request per key reload it (single flight in each server and a Redis lock across servers), make the others wait for that value, and add ±10% jitter to the 300 s TTL.

    Typical resultWhen a hot key expires, Postgres gets one read for it instead of hundreds, and keys no longer expire all at the same second.

    Seen on

    • Wikipedia: Describes a cache stampede: when a busy cached item expires, many requests try to rebuild it at once; one fix is a lock so only one of them recomputes it.
    • Go packages (singleflight): singleflight suppresses duplicate calls: only one call per key runs at a time, and a duplicate caller waits for it and gets the same result.

    You might describe it as

    • everyone misses the cache at the same moment
    • the database spikes every time a hot key expires
    • let one request refill the cache while the rest wait

    Not to be confused with

    • TTL (time to live)

      A TTL makes a key expire; a cache stampede is what happens when a busy key expires and every request reloads it at once.

    • Cache-aside

      Cache-aside is the normal routine of reading the cache, then the database on a miss; a cache stampede is that routine going wrong when many requests miss the same key together.