Design a Distributed Cache: System Design Interview Guide

Distributed cache design is one of the most common system design interview prompts at companies from Meta to Stripe to mid-size fintechs. Interviewers use it because it forces you to reason about trade-offs under real constraints: memory limits, network latency, node failures, and data consistency. This guide walks through how to structure your answer, the technology choices you should be ready to defend, and the follow-up questions interviewers love to ask.

Why Interviewers Ask This Question

A distributed cache sits between your application and a slower backing store (usually a database). It looks simple on the surface — “just store key-value pairs in memory” — but a strong candidate needs to cover scaling, failure handling, and staleness. Interviewers are grading your ability to navigate ambiguity, not your ability to recite Redis documentation.

Step 1: Clarify Requirements

Before touching a whiteboard, ask about:

• Read/write ratio. Caches are usually read-heavy (95%+ reads is common for product catalogs or session data).
• Data size. Are you caching small session tokens or large computed objects (e.g., search results)?
• Latency target. Sub-millisecond reads typically rule out anything that touches disk.
• Consistency tolerance. Can the client see stale data for a few seconds, or does every read need the latest write?

These answers determine whether you reach for Redis, Memcached, or a custom in-process cache with a distributed layer behind it.

Redis Cluster vs Memcached

This is the first fork in the road, and interviewers expect you to know the trade-offs cold.

Redis Cluster shards data automatically across nodes using hash slots (16,384 of them), supports replication with automatic failover, and offers rich data structures (sorted sets, hashes, streams) beyond simple key-value. It also supports persistence (RDB snapshots, AOF logs), which matters if the cache doubles as a lightweight source of truth for ephemeral data like leaderboards or rate limiters.

Memcached is simpler: pure in-memory key-value storage, multi-threaded per node, and historically faster for pure GET/SET workloads at very high throughput because it has less overhead. It has no built-in replication or persistence — if a node dies, that data is gone, and clients handle sharding themselves (or via a proxy like mcrouter).

The honest answer in an interview is: choose Memcached when you want the simplest possible cache for stateless, easily-repopulated data and raw throughput matters most. Choose Redis when you need richer data structures, built-in high availability, or want the cache to survive a restart.

Step 2: Choose a Caching Strategy

Cache-Aside (Lazy Loading)

The application checks the cache first. On a miss, it reads from the database, then writes the result into the cache. This is the most common pattern because it only caches what’s actually requested, and a cache outage degrades to “slower reads from the DB” rather than total failure.

Downside: the first request after an eviction or restart always pays the full database latency (cold cache problem), and there’s a window where the cache can hold stale data if the underlying row changes without an explicit invalidation.

Write-Through

Every write goes through the cache, which writes synchronously to the database before acknowledging. Reads are then always fast and consistent with what was last written through the cache. The cost is added write latency, since every write now blocks on two systems instead of one.

Write-Behind (Write-Back)

Writes land in the cache and are asynchronously flushed to the database later. This gives the lowest write latency but introduces risk: if the cache node crashes before the flush, you lose data. Most interview answers should mention this exists but flag it as risky for anything that isn’t easily reconstructible.

Comparison Table

Step 3: Handle Consistency and Invalidation

The hardest part of this design is keeping cache and database in sync without over-engineering. Common approaches:

• TTL-based expiration. Simple and robust — every key expires after N seconds regardless of writes. Good default for data where a few seconds of staleness is acceptable (e.g., product prices).
• Explicit invalidation on write. The write path deletes or updates the relevant cache key immediately. Requires more coordination but eliminates staleness windows.
• Versioned keys. Append a version number to cache keys tied to a row’s updated_at timestamp, so old cached values simply become unreachable once the version changes, without needing to actively delete anything.

Mention the thundering herd problem: when a hot key expires, hundreds of concurrent requests can all miss the cache simultaneously and hammer the database. The standard fix is a short-lived lock (or “request coalescing”) so only one request repopulates the cache while others wait.

Step 4: Scaling and Partitioning

Once a single node can’t hold your dataset, you need sharding. Two common approaches:

1. Consistent hashing distributes keys across nodes so that adding or removing a node only reshuffles a small fraction of keys, rather than a full rehash.
2. Proxy-based sharding (e.g., mcrouter, Redis Cluster’s own routing) hides shard topology from application code, which is more operationally friendly at scale.

If you want to go deeper on this specific slice, see our companion piece on Design a Distributed Cache: Architecture and Data Flow, which walks through consistent hashing, virtual nodes, and replication topology end to end.

Step 5: Failure Modes to Call Out

A strong candidate proactively names failure modes instead of waiting to be asked:

• Hot keys. A single celebrity user’s profile or a viral product page can overwhelm one shard even with good hashing. Mitigate with local (in-process) caching layered in front of the distributed cache, or key splitting.
• Cache stampede on cold start. After a full cache flush or new deployment, expect a temporary spike in DB load; consider pre-warming critical keys.
• Split-brain during network partitions. If a Redis primary is unreachable, a failover promotes a replica — but if the old primary comes back, you can briefly have two primaries accepting writes. Understanding this trade-off shows real depth.

Sample Interview Answer Structure

1. Clarify requirements (read/write ratio, size, latency, consistency needs) — 2 minutes.
2. Propose Redis Cluster or Memcached with justification — 2 minutes.
3. Pick cache-aside as the default strategy, mention write-through/write-behind as alternatives — 3 minutes.
4. Explain invalidation strategy and thundering herd mitigation — 3 minutes.
5. Address scaling via consistent hashing and sharding — 3 minutes.
6. Proactively discuss failure modes — 2 minutes.

This structure covers roughly 15 minutes, leaving room for interviewer follow-ups, which is the pacing most FAANG-style interviews expect.

Common Follow-Up Questions

• “What happens if a cache node goes down mid-request?” — Discuss client-side retry with a different replica, or graceful degradation to the database.
• “How would you cache a leaderboard that updates every second?” — This pushes toward Redis’s sorted set data structure rather than plain key-value.
• “How do you prevent cache poisoning from a buggy write path?” — Discuss short TTLs as a safety net even when you have explicit invalidation.

Practice More

If you want structured practice for this and dozens of other recurring system design prompts, along with behavioral and coding-round frameworks, The 0-to-1 SWE Interview Playbook (Amazon: https://www.amazon.com/dp/B0H256Z1MF?tag=sirjohnnymai-20) walks through worked examples with the same level of interviewer-perspective detail as this guide.

Key Takeaways

• Always clarify read/write ratio and consistency tolerance before naming a technology.
• Cache-aside is the safe default; write-through and write-behind are situational.
• TTLs plus explicit invalidation cover most consistency needs without overengineering.
• Consistent hashing and request coalescing are the two concepts that separate strong answers from great ones.