Design a Search Autocomplete: Architecture and Data Flow

Once you’ve settled on a trie as the core data structure for a search autocomplete system, the interview shifts to a harder question: how do all the pieces fit together in production? This is where candidates who’ve only memorized “use a trie” start to struggle, and where candidates who understand data flow pull ahead. This article walks through the full architecture, component by component.

The Four Core Components

Every production autocomplete system, regardless of company, breaks down into four pieces:

1. Data collection — capturing what users actually search for.
2. Aggregation pipeline — turning raw logs into ranked suggestion data.
3. Trie service — the in-memory structure that serves prefix lookups.
4. Serving/API layer — the stateless layer clients actually talk to.

Let’s walk through each.

Data Collection: Capturing Query Signal

Every search a user submits (not every keystroke, just the final submitted query) gets logged as an event: {query: "system design interview", user_id, timestamp, region}. These events stream into a durable log, typically Kafka or a managed equivalent (Kinesis, Pub/Sub).

Design considerations at this stage:

• Sampling. At extreme scale (billions of queries/day), you may sample a percentage of traffic rather than logging everything, trading some ranking precision for reduced infrastructure cost.
• Privacy. Query logs often contain sensitive information. Hashing or anonymizing user identifiers before they enter the aggregation pipeline is worth mentioning even if the interviewer doesn’t ask.
• Deduplication. Bot traffic and accidental double-submits can skew frequency counts; a lightweight dedup filter (same user, same query, within a short window) cleans this up before aggregation.

Aggregation Pipeline: From Logs to Ranked Data

The aggregation pipeline runs as a batch or micro-batch job (Spark, Flink, or a simpler cron-triggered job for smaller scale) on a fixed cadence, commonly every 5-15 minutes.

Its job:

• Count query frequency over a rolling window (e.g., last 7 days, weighted toward recent days).
• Apply a decay function so old spikes fade out gracefully instead of dominating forever.
• Filter out low-quality or malicious queries (single-character spam, injection attempts).
• Emit a sorted list of (query, score) pairs, partitioned however makes sense (by first letter, by region, etc.) to parallelize the next step.

The output of this stage is a trie build job: a separate process reads the ranked query list and constructs a new trie structure, including the top-N cache at each node discussed in data-structure design. This build can take anywhere from seconds to minutes depending on corpus size, which is why it happens offline rather than inline with user requests.

The Trie Service: Serving Layer for Prefix Lookups

Once built, the trie is serialized (often as a flat array-based representation rather than pointer-based nodes, for cache efficiency and fast loading) and pushed to the trie-serving fleet.

Key architectural decisions here:

• Full replication vs. sharding. For most corpora (tens of millions of queries), the entire trie fits in memory on a single machine, so the simplest design replicates the full trie across every serving node behind a load balancer — no sharding needed. Sharding by first character only becomes necessary at truly massive scale (hundreds of millions of distinct query strings) and adds real complexity (fan-out queries, merge logic), so justify it before proposing it.
• Blue-green snapshot swaps. Each serving node keeps two trie instances in memory: the currently active one and the next one being loaded. Once the new trie finishes loading and passes a sanity check (e.g., non-empty, size within expected bounds), an atomic pointer swap makes it live. This avoids any query ever hitting a half-built structure.
• Health checks and rollback. If a newly built trie fails validation (corrupted data, wildly different size than expected), the serving node keeps using the previous good snapshot and alerts on the anomaly.

The Serving/API Layer

Clients don’t talk to trie nodes directly. A thin, stateless API layer sits in front:

• Receives the prefix from the client (usually via a debounced request — client waits ~150-200ms after the last keystroke before firing, to avoid flooding the backend on every character).
• Routes the request to a trie-serving node (simple round-robin or least-connections load balancing).
• Applies any lightweight, request-time re-ranking (e.g., boosting results matching the user’s locale).
• Returns the top 5-10 suggestions as JSON.

This layer is also where caching pays off. A CDN or edge cache (or even a simple in-memory LRU cache in the API layer) for extremely common prefixes — single letters, top brand names — can absorb a large fraction of total traffic before it ever reaches a trie node.

Full Data Flow, End to End

Putting it together: a user types a prefix, the client debounces and sends it to the API layer, which forwards it to a trie-serving node holding the current in-memory trie snapshot, which returns ranked completions in O(k) time. Meanwhile, completed searches flow asynchronously into the logging pipeline, get aggregated on a 5-15 minute cadence, produce a new ranked query list, get built into a fresh trie, and get pushed out to serving nodes via a blue-green swap. The read path and write path never block each other.

Comparison: Architectural Choices and Their Tradeoffs

Failure Modes to Discuss

Interviewers reward candidates who proactively raise failure scenarios:

• Aggregation pipeline falls behind. If the batch job lags, suggestions get stale but the system stays up — graceful degradation, not an outage. Worth calling out explicitly.
• Serving node crash. Load balancer health checks route around it; the node reloads the last good trie snapshot on restart.
• Cold start. A brand-new region or product line has no query history yet. A fallback to a curated seed list (manually configured top queries) avoids an empty-state experience.

Wrapping Up

The architecture behind search autocomplete is a clean separation between a slow-moving batch pipeline (data collection, aggregation, trie building) and a fast, stateless serving path (trie lookup, light re-ranking, response). Once you can draw this diagram cleanly and explain why each boundary exists, you’ve demonstrated the kind of systems thinking interviewers are actually screening for.

For a structured framework you can reuse across dozens of system design prompts beyond autocomplete, see The 0-to-1 SWE Interview Playbook (Amazon: https://www.amazon.com/dp/B0H256Z1MF?tag=sirjohnnymai-20), which breaks down this exact read-path/write-path separation pattern as a reusable interview tool.