· SWE Editorial · System Design · 5 min read
Design a Rate Limiter: System Design Interview Guide
Rate limiter questions look simple and punish candidates who treat them that way. This guide covers the four algorithms interviewers expect you to know — token bucket, sliding window, fixed window counter — and how to build a distributed, Redis-backed version that survives a real follow-up.
Why this question shows up in almost every loop
Rate limiters are one of the most commonly asked system design questions because they’re small enough to fully design in 45 minutes and deep enough to reveal how a candidate thinks about tradeoffs. The bar isn’t “can you block requests over N per second” — it’s whether you can articulate which algorithm fits which traffic pattern, and whether the thing works when it’s not running on a single machine.
This guide is the interview-prep version: what to say, in what order, and what follow-up questions to expect.
Step 1: clarify the requirements
Before touching an algorithm, nail down:
- Scope of limiting: per-user, per-IP, per-API-key, or global?
- Limit shape: “100 requests per minute” vs. “burst of 20 then steady 5/sec”?
- Behavior on limit exceeded: hard reject (429) vs. queue vs. throttle?
- Where does it run: API gateway, middleware in each service, or a dedicated sidecar?
Interviewers notice when candidates skip this and jump straight to “I’ll use a token bucket.” Ask first.
Step 2: the four core algorithms
Fixed window counter
Divide time into fixed windows (e.g., 00:00-00:59, 01:00-01:59) and count requests per window per key. Reset the counter at window boundaries.
- Pro: trivial to implement, O(1) memory per key.
- Con: burst-at-boundary problem — a client can send the full limit right before a window ends and the full limit again right after, doubling the effective rate in a short span.
Sliding window log
Store a timestamp for every request in a sorted set per key. On each new request, drop timestamps older than the window and count what’s left.
- Pro: exact, no boundary problem.
- Con: memory scales with request volume — a high-traffic key can store thousands of timestamps. Expensive at scale.
Sliding window counter (approximation)
Combine fixed windows with weighted interpolation: estimate the current rate as a weighted average of the previous window’s count and the current window’s count, weighted by how far into the current window you are.
- Pro: near-exact accuracy with O(1) memory — the practical default for most production systems.
- Con: slightly approximate; edge cases with highly irregular traffic can drift.
Token bucket
Each key has a bucket with capacity C, refilled at rate R tokens/sec. Each request consumes one token; if the bucket is empty, reject or delay.
- Pro: naturally supports bursts up to bucket capacity while enforcing a long-run average rate — closest to how real traffic actually behaves.
- Con: slightly more state to track (current token count + last refill timestamp) than a fixed counter.
| Algorithm | Memory per key | Burst handling | Accuracy | Typical use |
|---|---|---|---|---|
| Fixed window | O(1) | Poor (boundary spikes) | Approximate | Simple internal limits |
| Sliding window log | O(N) requests | Exact | Exact | Low-volume, high-precision needs |
| Sliding window counter | O(1) | Good | Near-exact | Most production APIs |
| Token bucket | O(1) | Best (allows controlled bursts) | Exact for its model | Public APIs, user-facing limits |
For a system design interview, token bucket is the default answer unless the requirements specifically call for something else — say so, then explain why (burst tolerance without sacrificing long-run enforcement), then mention sliding window counter as the alternative if strict burst prevention matters more than burst tolerance.
Step 3: making it distributed
A rate limiter that only works on one server is a toy. The moment you have multiple API servers behind a load balancer, in-process counters are wrong — a user hitting server A and server B independently doubles their effective limit.
The standard fix: centralize state in Redis.
Client -> Load Balancer -> App Server 1 --\
-> App Server 2 ---> Redis (shared counters/buckets)
-> App Server N --/Each app server, on a request, does an atomic read-modify-write against Redis (via INCR + EXPIRE for fixed window, or a Lua script for token bucket to make the check-and-decrement atomic). This is the natural bridge into the “distributed rate limiting” follow-up, which the companion article on distributed implementation covers in depth — Lua scripts, race conditions, and multi-region consistency.
Step 4: where does the limiter live
- API Gateway / edge (e.g., Kong, Envoy, Cloudflare): centralizes the logic, protects all downstream services uniformly, but adds a network hop and a shared dependency.
- Middleware in each service: more flexible per-endpoint limits, but duplicated logic and configuration across services.
- Dedicated rate-limiting microservice: called synchronously by other services before processing a request — clean separation, but adds latency and a new single point of failure unless it’s made highly available.
Most real systems land on API gateway for coarse, global limits (per-IP, per-API-key) plus per-service middleware for fine-grained business-logic limits (e.g., “3 password reset emails per hour per account”).
Step 5: what interviewers probe next
Once the base design is on the board, expect:
- “What happens if Redis goes down?” — good answers mention fail-open vs. fail-closed tradeoffs, and local in-memory fallback limiters as a degraded mode.
- “How do you handle a user hitting two different regions?” — this is the multi-region consistency question; a reasonable answer accepts eventual consistency for rate limits (slight over-admission is usually an acceptable tradeoff vs. the cost of strict global consensus).
- “How would you rate-limit by cost, not by request count?” (e.g., GraphQL queries of varying complexity) — token bucket generalizes naturally here: consume N tokens per request instead of 1, where N is a computed cost.
The interview-ready summary
If you only remember one structure for this question: clarify requirements → pick token bucket (or sliding window counter) with a one-sentence justification → centralize state in Redis for multi-server correctness → name the gateway-vs-middleware placement tradeoff → proactively raise the Redis-failure and multi-region questions before you’re asked. That sequence, delivered in under 15 minutes, reads as senior-level regardless of which specific algorithm you land on.
Further reading
The 0-to-1 SWE Interview Playbook (Amazon: https://www.amazon.com/dp/B0H256Z1MF?tag=sirjohnnymai-20) walks through this exact rate limiter framework alongside URL shorteners, chat systems, and news feed design — with the specific follow-up questions interviewers ask after each core answer, so you’re never caught flat-footed on the second question.