Course · 04

How does a cache make things fast, and how does it lie to you?

A cache keeps a copy of data close by, so you skip the slow trip. This page covers where caches live, how they fill and empty, how they go wrong, how HTTP and CDNs cache, and the math behind a hit ratio.

—Words you need

Each jargon word once, in plain words glossary

01Why cache at all

Skip the slow trip, protect the slow thing core

Reading from memory is much faster than reading from disk or over a network. A database query also costs CPU, locks and connections. If many people read the same thing, it is wasteful to compute it again each time.

A cache keeps the answer. It does two jobs. It makes reads faster for users. It also protects the database from load it could not handle.

Caching works when reads are far more common than writes, and when a little staleness is fine. It fails when every read is unique, or when data must be exact to the millisecond.

A cache is a bet that the same data will be read again soon. Check the bet with the hit ratio.

Source: Azure Architecture Center: Cache-Aside

02Where caches live

Six layers, from the user's browser down to the database core

An in-process cache is fastest but each server has its own copy. A distributed cache is one shared copy, one network hop away.

Source: MySQL buffer pool · Azure Cache-Aside (local vs shared) · RFC 9111

03Cache-aside (lazy loading)

The default pattern: the app manages the cache itself interview favourite

Read: try the cache. On a miss, read the DB, put the value in the cache with a TTL, and return it.

Write: update the DB first. Then delete the cache key. The next read fills it again.

def get_user(id):
    v = cache.get(f"user:{id}")
    if v is not None:
        return v                                   # hit
    v = db.query("SELECT * FROM users WHERE id=%s", id)  # miss
    cache.set(f"user:{id}", v, ttl=300)
    return v

def update_user(id, fields):
    db.update("users", id, fields)                 # 1. source of truth first
    cache.delete(f"user:{id}")                     # 2. then invalidate

Cache-aside is the safe default: DB first, then delete the key, and always set a TTL as a safety net.

Source: AWS ElastiCache strategies · Azure Cache-Aside

04Read-through, write-through, write-behind, write-around

Who fills the cache, and when the database gets the write core

Read-through. Like cache-aside, but the cache library loads from the DB itself on a miss. The app only talks to the cache.

Write-around. Writes go straight to the DB and skip the cache. The cache fills only when someone reads. Good for data that is written once and rarely read, like logs.

The write pattern decides the freshness and the risk: write-through pays on writes, write-behind risks loss, cache-aside accepts brief staleness.

Source: AWS ElastiCache strategies · Azure Cache-Aside

05Eviction: LRU, LFU, FIFO, TTL

What to throw out when memory is full core

A cache is smaller than the data. When it is full and a new item comes, something must go. The eviction policy picks the victim.

get A   →  [A]
get B   →  [A, B]
get C   →  [A, B, C]
get A   →  [B, C, A]        # A used again, moves to newest
get D   →  [C, A, D]        evict B: least recently used
get C   →  [A, D, C]

Real LRU needs a hash map plus a doubly linked list. Both give O(1) get and put: the map finds the node, the list moves it to the front.

For a pure cache, set maxmemory and use allkeys-lru or allkeys-lfu. Never leave a cache on noeviction.

Source: Redis key eviction · MySQL buffer pool (scan-resistant LRU)

06Invalidation, stale data and consistency

The famous hard problem, step by step deep dive

A cache holds a copy. When the real data changes, the copy is now wrong. Invalidation is how you remove or fix it. There are three tools.

Why delete instead of update? Two writers can update the cache in the wrong order and leave the old value. A delete is safe to repeat; it is idempotent (doing it twice equals doing it once).

A real race in cache-aside

Fixes: a short TTL limits the damage. A delayed second delete ("delete again after a short wait") catches late sets; this is a common practice, not a standard. Leases, below, close the gap properly.

Delete, do not update; always keep a TTL; and if staleness really matters, use leases or versioned values.

Source: Scaling Memcache at Facebook (NSDI 2013) · Azure Cache-Aside (consistency)

07Stampede, penetration, avalanche, hot keys

Four ways a cache sends a flood to the database interview favourite

Cache stampede (thundering herd)

A hot key expires. A thousand requests miss at the same moment. All thousand run the same slow query. The DB falls over.

if now - delta * beta * log(random()) >= expiry:   # log(random()) is negative
    recompute_and_reset()                           # chance rises as expiry nears

Cache penetration

Requests ask for keys that do not exist, like user:-1. Every one misses and hits the DB. Attackers can do this on purpose.

Fixes: negative caching (store "not found" with a short TTL) and a Bloom filter of all real IDs. A Bloom filter can say "surely not here" with no false negatives, so you skip the DB.

Cache avalanche

Many keys expire at the same moment, often because they were loaded together with the same TTL. Or the whole cache restarts empty. The DB gets everything at once.

Fixes: add jitter (random extra seconds) to TTLs, warm the cache before traffic, use replicas so one node loss does not empty it, and rate limit the DB path.

Hot keys

One key gets a huge share of traffic, like a celebrity's profile. It lives on one cache node, so that node melts while others are idle.

Fixes (common practice, not one standard): a small in-process cache in front for that key, copies of the key under suffixes (key#1 to key#8) read at random, and read replicas.

Every cache failure mode ends the same way, a flood on the DB. Coalesce misses, cache "not found", jitter TTLs, and spread hot keys.

Source: Memcache at Facebook · AWS Builders' Library: Caching challenges · Vattani et al., VLDB 2015 · Redis Bloom filter

08HTTP caching headers

How a server tells browsers and CDNs what they may keep core

HTTP has caching built in. The server adds headers to the response. Every cache on the path, browser or CDN, reads them and obeys.

# first visit
GET /app.css                       →  200 OK
                                       Cache-Control: max-age=600
                                       ETag: "v42"
# 15 minutes later: stale, so the browser asks "is v42 still right?"
GET /app.css
If-None-Match: "v42"               →  304 Not Modified   (no body, reuse the copy)

Version your static files and give them a long max-age with immutable; give HTML no-cache plus an ETag.

Source: RFC 9111 HTTP Caching · RFC 5861 · MDN Cache-Control · MDN ETag

09CDNs and how CloudFront works

Copies of your files in cities near your users deep dive

A CDN (content delivery network) is many cache servers around the world. The edge is the site near the user. The origin is your real server or bucket.

Keep the cache key small, version your file names, and treat invalidation as an emergency tool, not a deploy step.

Source: CloudFront: how it works · CloudFront: cache key · CloudFront: invalidation pricing · CloudFront: expiration · CloudFront: Origin Shield

10Cache hit ratio math

One worked example you can redo in an interview worked example

Hit ratio = hits ÷ (hits + misses). The numbers below are an example for practice, not a measurement.

DB reads/s    = 10,000 × (1 − 0.90)            = 1,000
avg latency   = 0.90 × 1 ms + 0.10 × (1 + 10) ms
              = 0.9 + 1.1                     = 2.0 ms     # a miss pays the cache check too

at 99% hits:  DB reads/s = 100,   avg = 0.99 × 1 + 0.01 × 11 = 1.1 ms

Look at the jump from 90% to 99%. The hit ratio rose 9 points, but DB load fell 10 times. Near the top, every point matters.

Size the DB for the miss traffic, and remember the miss traffic jumps the moment the cache is cold.

Source: Redis eviction docs (INFO stats) · math is a worked example on this page

—When to use what

Situation, pick, why decision table

—Common mistakes and traps

Things that look right and break in production traps

—Interview questions

20 questions, easy to design-level. Try first, then open the answer drill

—Cheat sheet

Numbers and facts to remember memorise

—Sources

All checked 2026-10-25 references