arrow_backBack to Phase 2
bolt

Caching
Strategies

The fastest network request is the one you don't make. Caching is borrowing RAM from the future to pay for the latency of the past.

Why We Cache

Temporal Locality

"If I just read this data, I'm likely to read it again soon."

Example: A viral tweet being refreshed by thousands of users.

Spatial Locality

"If I read this data, I'm likely to read its neighbors soon."

Example: Loading a user profile → likely need their recent posts too.

Caching Patterns

1

Cache-Aside (Lazy Loading)

The application is responsible for reading and writing to the cache. The cache doesn't know about the database.

  1. App checks Cache.
  2. If Miss: App reads DB.
  3. App writes to Cache.
  4. App returns data.
Pros:
  • Only requested data is cached.
  • Cache failure doesn't kill the app (just slower).
Cons:
  • Cache Miss penalty (3 trips).
  • Stale data if DB updates without cache invalidation.
2

Write-Through

App writes to the Cache, and the Cache synchronously writes to the DB.

  1. App writes data to Cache.
  2. Cache writes to DB.
  3. Both return success.
Pros:
  • Data in cache is never stale.
  • Read performance is excellent.
Cons:
  • Write latency is higher (2 writes).
  • Cache pollution (writing data that's never read).
3

Write-Back (Write-Behind)

App writes to Cache and returns immediately. Cache asynchronously writes to DB later.

  1. App writes to Cache.
  2. App gets "Success".
  3. Cache updates DB in background.
Pros:
  • Lowest write latency.
  • Good for write-heavy workloads.
Cons:
  • Data Loss Risk: If cache crashes before DB sync.
  • Complex implementation.

Eviction Policies

Cache memory is expensive and finite. When it's full, who gets kicked out?

LRU

Most Common

Least Recently Used

Discards the item that hasn't been used for the longest time.

Good for: General purpose, Recency bias.

LFU

Usage Based

Least Frequently Used

Counts how often an item is needed. Kicks out the unpopular ones.

Good for: Stable access patterns.

FIFO

Simple

First In First Out

Oldest item leaves first, regardless of usage.

Good for: Sequential data.

TTL

Time Based

Time To Live

Items expire after a set time (e.g., 5 mins).

Good for: Data that must be fresh (prices, inventory).

The Thundering Herd

warning

The Scenario

A popular cache key expires (or cache server crashes). Suddenly, 10,000 requests hit the cache at the same time, get a MISS, and ALL hit the Database simultaneously.

Result: Database CPU spikes to 100%, System goes down.

The Fixes

Randomized TTL

Don't expire all keys at exactly 5:00 PM. Add jitter (e.g., 5 min ± 30s).

Request Coalescing

If 100 requests want key 'A', let one go to DB, making the other 99 wait for that result.

Distributed Caching

When your cache is bigger than one server's RAM, you need a cluster (Redis Cluster, Memcached).

How to find the key?

If you have 5 cache servers, where does key "user_123" live?

Modulo Hashing
hash("user_123") % 5 = Server 2

Bad because adding a server breaks 100% of mappings.

Consistent Hashing
Map to a ring. Find next server clockwise.

Good! Adding a server only reshuffles 1/N keys.

Redis vs Memcached

  • RedisSingle-threaded event loop. Supports complex data structures (Lists, Sets, Sorted Sets). Persistence options.
  • MemcachedMulti-threaded. Pure Key-Value store. Simple, extremely high throughput for simple strings.