A creator posts a video. It’s good, so it’s shared, and by the time you notice the graphs it’s being opened forty thousand times a second. Every open needs the same handful of rows — the creator’s profile, the video metadata, the follower count. Your database, which was comfortable at a few thousand reads a second, is now serving the identical SELECT forty thousand times a second and its p99 is climbing through the roof.
Nothing about the data got harder. The same rows are being read over and over. The database is just the wrong place to answer a question you already answered a microsecond ago. That’s the entire premise of a cache: put the answer somewhere fast and close, and stop asking the slow, far thing.
Why bother: latency and offload, in that order
Two things a cache buys you, and they’re worth separating because interviews conflate them.
Latency. A read from Redis in the same datacenter is sub-millisecond; a read from a spinning disk or a cross-region database is orders of magnitude slower. Amazon famously found that every extra 100ms of latency cost them ~1% in sales — speed is the product.
Offload. This is the one that saves you at 3am. The cache doesn’t just make the fast requests faster; it absorbs the requests so they never reach the origin. A 95% hit rate means your database sees 5% of the traffic. That’s the difference between a viral post being a graph you screenshot and a graph that pages you.
The shape: client → cache → origin
Every caching setup is the same three boxes with two paths between them — a hit that stops at the cache, and a miss that falls through to the origin and (usually) backfills on the way home.
Where that backfill logic lives — in your application code or inside the cache — and when writes touch the cache versus the origin, is what the named strategies below are actually about.
Read patterns: who fills the cache on a miss
Cache-aside (lazy loading). The application owns the logic. On a read it checks the cache; on a miss it reads the origin, writes the result back, and returns it. The cache is a dumb key-value box that knows nothing about your database. This is the default — it’s what most people mean by “we added Redis” — because only data that’s actually requested ever gets cached, and a cache outage degrades to slow, not broken.
Read-through. The cache itself sits in front of the origin and loads on a miss, via a provider you configure. Your application only ever talks to the cache; it never sees the origin. Cleaner call sites, but you’ve handed the read path to a library and a cold cache stampedes the origin unless it coalesces (more on that below).
Write patterns: the consistency/latency dial
Where a write lands — and whether the caller waits for the origin — is the real design decision. It’s a dial between fast writes and consistent reads.
- Write hits cache AND origin synchronously, in one call
- Cache is never staler than the last write
- Reads after a write are always correct
- Cost: every write pays the origin latency; you cache data that may never be read
- Write hits cache only; origin updated async, batched
- Writes are cache-fast; great for write-heavy, bursty loads
- Coalesces N writes to one key into one origin write
- Cost: a cache crash before flush LOSES data; origin is eventually consistent
Two more you should be able to name:
- Write-around. Writes go straight to the origin and skip the cache; the cache is populated only on a subsequent read (cache-aside style). Right when written data is rarely read soon after — bulk imports, logs — so you don’t evict hot keys to cache write-only data. Cost: a read right after a write is a guaranteed miss.
- Write-back is not write-through with a delay — it changes your durability story. If losing the last few seconds of writes on a cache crash is unacceptable (payments, inventory), write-back is off the table no matter how tempting the latency is.
Eviction: the cache is smaller than the truth
A cache is deliberately smaller than the origin, so it’s always full and always evicting. The policy decides what to drop when a new key needs room — and the right one depends on your access shape.
- LRU — evict least-recently-used. The default. Great when recent = relevant (sessions, feeds).
- FIFO — evict oldest-inserted, ignores reads. Cheap, but drops hot keys that were loaded early.
- TTL — expire after a fixed lifetime. Orthogonal to the above: bounds staleness, not size.
- LFU — evict least-frequently-used. Wins when a small set of keys is hot forever (a viral video).
- LFU resists one-off scans that would flush an LRU cache (a crawler reading every key once).
- Cost: needs frequency counts; a once-hot-now-cold key can linger. Redis LFU decays counts to fix this.
TTL is the one people underuse: even with a perfect LRU/LFU policy, a TTL is your ceiling on staleness — the guarantee that a wrongly-cached or forgotten-to-invalidate value can’t live forever.
The hard part: invalidation
There are only two hard things in Computer Science: cache invalidation and naming things. — Phil Karlton
Filling a cache is easy. Knowing when the cached copy is now a lie — because someone updated the origin behind the cache’s back — is the problem that has no clean answer. Two broad moves: TTL (accept bounded staleness — the value auto-expires) and explicit invalidation (on write, delete or update the key). TTL is simple and self-healing but always stale up to the TTL; explicit invalidation is fresh but only as correct as your ability to find every key derived from the changed data — and in a system with derived, composite, and fanned-out cache keys, that’s genuinely hard.
Failure modes interviews love
Caches don’t fail quietly — they fail by suddenly not being there, and shoving their entire absorbed load onto an origin that hasn’t seen it in weeks.
Cache penetration is the mirror image: requests for keys that don’t exist in the origin either (a scan for random user IDs, an attack). Every one misses the cache, misses the DB, and returns nothing — so the cache never fills and every request is a DB hit. Fix: negative caching (cache the “not found” as a short-lived tombstone) and/or a bloom filter in front that answers “definitely not present” in O(1) without touching the DB — see Bloom Filters for how that probabilistic gate works.
Hot keys — one key so popular it saturates a single cache node’s CPU or network (the viral creator’s profile). No eviction policy helps; you replicate the hot key across nodes, or add a small local (in-process) cache in front of the shared one so most reads never leave the box.
The code: cache-aside, stampede-proofed
Cache-aside is four lines until you make it safe under load. Here it is with single-flight coalescing, negative caching, and a jittered TTL — the three mitigations that turn a naive read path into one that survives going viral.
Read it top to bottom: the fast path returns on a hit; a miss takes a per-key lock so a herd collapses to one origin read; a missing row is cached as a tombstone so a non-existent key can’t be a permanent DB hole; and every TTL is jittered so a cohort of keys can’t expire in the same second and stampede together.
In the wild
Interview corner
Check yourself
References
Official documentation
- Redis — eviction policies (
maxmemory-policy:allkeys-lru,allkeys-lfu,volatile-ttl) and the LFU count-decay design. - Memcached — LRU eviction and the multithreaded key-value model (memcached.org).
- AWS — “Caching strategies” (ElastiCache developer guide): cache-aside (lazy loading), write-through, and TTL guidance.
RFCs / Papers
- Nishtala et al., Scaling Memcache at Facebook, NSDI 2013 — leases, the thundering-herd mitigation, and negative caching at scale.
- Vattani, Chierichetti, Lowenstein, Optimal Probabilistic Cache Stampede Prevention, VLDB 2015 — the
XFetchprobabilistic early-recompute algorithm.
Engineering blogs
- Cloudflare / Fastly — CDN edge caching and cache-key/invalidation design.
Books
- Martin Kleppmann — Designing Data-Intensive Applications, on derived data, caches, and consistency.
Discussion
Loading the conversation…
Discussion
Loading the conversation…