13. Search Autocomplete / Typeahead Intermediate → Advanced
A. Realistic Interview Prompt
Every character a user types is, underneath, a fresh request for "rank and return the best few completions for this exact prefix," fired at keystroke speed, at a volume that dwarfs the number of searches actually submitted — a single search is preceded by several throwaway prefix requests that never turn into anything. The honest, load-bearing answer this problem is actually testing is one integrated decision: a Trie (prefix tree) with the top-K completions precomputed and cached at each node, rebuilt periodically offline from aggregated query logs, with a fast-moving "trending" overlay merged on top of that otherwise-stale structure, sharded by prefix range once the corpus outgrows one machine, and aggressively cached at the edge for the small set of short, extremely hot prefixes that account for a disproportionate share of all traffic.
B. Difficulty
Intermediate, growing into genuine distributed-systems territory in the deep dives — the read path itself (walk a tree, return a cached list) is conceptually simple once you know to reach for a trie at all; what makes this problem hard is everything around that read: keeping the structure fresh without paying a write cost on every keystroke (Patterns Playbook C), sharding a tree-shaped data structure rather than a flat keyspace (Patterns Playbook G), and layering personalization and typo-tolerance on top without breaking the one thing that makes the whole design fast — a shared, cacheable, precomputed structure.
C. What This Problem Tests
- Recognizing that the naive, "obvious" implementation — a live database prefix scan, or an in-memory trie that recomputes ranking on every request — breaks specifically on the hottest, shortest prefixes, which is exactly backwards from what a scalable design needs
- Precomputing expensive work off the hot path (Patterns Playbook C) as the central move, generalized here from "defer a side effect to a queue" to "defer an entire ranking computation to an offline batch job" — the same instinct behind News Feed's precomputed
FeedEntrystore (P3, Deep Dive O) - Sharding a data structure whose natural ownership boundary is a tree's own branches, not an entity ID — the same "shard by the natural ownership key" reasoning as Patterns Playbook G, just applied to prefix ranges instead of a row's primary key
- Distinguishing a slow-changing, globally-shared bulk structure from a fast-moving, narrow "what's trending right now" overlay — two different freshness requirements that need two different mechanisms, not one compromise setting
- Recognizing this as a read-heavy, staleness-tolerant system from the start (P11's caching reasoning applies almost unchanged) and being explicit about where personalization and fuzzy-matching can and can't be bolted onto a precomputed, shared cache without destroying its economics
D. Requirements
Functional
- Given a partial search string (a prefix), return a ranked list of the top-K most likely completions
- Ranking reflects overall popularity across all users — how often a completion has actually been searched — not simple alphabetical order
- Suggestions update, on some reasonable delay, to reflect genuinely new or newly-trending queries, not just what was popular when the system was first built
- Results are returned per keystroke, incrementally, as the user's prefix grows or shrinks
- (Extension, addressed in R/S) results can be personalized per user and tolerant of minor typos
Non-Functional
- Very low latency — suggestions need to feel instantaneous relative to typing speed; a response arriving after the user has already typed the next character is functionally useless
- Extremely read-heavy, at very high volume — every keystroke is a request, and only a small fraction of prefix requests ever culminate in an actual submitted search
- Highly skewed traffic — a small number of short, common prefixes ("a", "how", "wea...") receive a wildly disproportionate share of total request volume relative to long, specific ones
- Staleness-tolerant — a few minutes (or, for the bulk of the ranking, longer) of lag between "this query became popular" and "this query shows up in suggestions" is an acceptable, deliberate trade-off, not a correctness failure (elaborated fully in V)
- Horizontally scalable storage — the full ranked corpus of query completions does not need to, and should not be assumed to, fit on one machine
Out of scope: full-text search over document contents (this system only autocompletes search-box query strings, not searching within results); a complete natural-language query-understanding layer; building a production-grade spelling-correction model from scratch (S names the mitigations available without claiming to design one in the room).
E. Clarifying Questions
| # | Question | Why It Matters | Design Decision Affected |
|---|---|---|---|
| 1 | Does ranking need to reflect real-time trending activity (breaking news, a sudden spike), or is "popular over the last several days/weeks" an acceptable basis for ranking? | Determines whether a fast-moving real-time overlay is a first-class requirement or a later extension | Deep Dive 3 (O), Capacity Estimation (G) |
| 2 | Should suggestions be personalized to an individual user's own search history, or is one shared, global ranking sufficient? | Changes whether personalization is a core requirement shaping the read path from the start, or a bolt-on layer | Deep Dive 6 (R) |
| 3 | Do we need to tolerate typos/misspellings in the prefix itself, or is exact-prefix matching acceptable? | An exact-prefix trie structurally cannot answer a typo-tolerant query on its own — this changes whether a secondary mechanism is required at all | Deep Dive 7 (S) |
| 4 | Roughly how large is the corpus of distinct query strings we're ranking over, and how long can completions realistically be? | Directly drives whether the precomputed structure fits comfortably on one machine or requires sharding from the start | Capacity Estimation (G), Deep Dive 4 (P) |
| 5 | Is this general web-search autocomplete, or scoped to a narrower domain (product search, a company directory)? | A narrower, smaller corpus may never need sharding at all — worth confirming before assuming the full-scale version is required | Assumptions (F), Capacity Estimation (G) |
F. Assumptions
- Confirmed: ranked, per-keystroke prefix completion, ranked by real popularity rather than alphabetically, with results that update over time as query trends shift.
- Assumption: this is general, large-scale web-search autocomplete (E's clarifying question 5) — the assumption that makes sharding (P) and edge caching (Q) first-class parts of the design rather than something a smaller-scoped system would never need.
- Assumption: both a slower, broad popularity ranking and a fast-moving trending signal are required (E's clarifying question 1) — treated throughout as two distinct mechanisms with two distinct freshness targets, never conflated into one setting.
- Assumption: personalization (E's clarifying question 2) and typo tolerance (E's clarifying question 3) are both real, named extensions to the core design (R, S) rather than requirements the base trie itself must satisfy directly.
- Assumption: the corpus of distinct query strings being ranked over is large enough (hundreds of millions of frequently-seen strings) that it will not comfortably fit in one machine's memory once precomputed rankings are included (E's clarifying question 4) — this is the assumption that makes sharding (P) load-bearing rather than a hypothetical extension.
G. Capacity Estimation
| Metric | Value / Calculation | Why It Matters |
|---|---|---|
| Total submitted searches | Illustratively, ~1B searches/day platform-wide | The baseline "real" query volume — everything else in this table is a multiple of this number, not a replacement for it |
| Autocomplete request volume | Each submitted search is preceded by roughly 5–7 keystroke-triggered prefix requests (after client-side debouncing) → ~6–7B autocomplete requests/day ≈ ~75,000/sec average | This, not the search-submission rate, is the number the read path has to be sized for — the actual dominant traffic in this entire system |
| Peak traffic skew | Peak hours plus the natural skew toward short/common prefixes push the effective rate on hot paths to ~4–5x average during peak windows | Confirms this is fundamentally a read-fan-in problem concentrated on a small number of very hot prefixes, not an evenly-spread load |
| Distinct query corpus after filtering | Illustratively, ~100–200M frequently-seen distinct query strings survive the offline aggregation job's noise/long-tail filtering (out of a much larger raw logged corpus) | Sets the outer bound on trie node count — everything below this line filters out the one-off, rarely-repeated tail that isn't worth precomputing ranking for at all |
| Trie memory footprint | ~200M shared-prefix nodes × ~500–600 bytes/node (children pointers + a small top-10 cached completion list per node) ≈ roughly 100–120 GB for the full structure | Comfortably beyond a single machine's safe in-memory budget — the concrete number that makes sharding (P) a necessity, not a hypothetical |
| Per-request latency budget | Sub-100ms end to end, ideally well under that, to feel instantaneous relative to typing speed | Rules out anything that scans, joins, or re-ranks a large candidate set live per request — the entire justification for precomputing top-K (N) ahead of time |
H. Core Entities
| Entity | Purpose | Key Fields |
|---|---|---|
QueryLogEntry | One raw record of a search actually submitted by a user, the ultimate source of all ranking signal | query_text, user_id (optional), timestamp, region/locale, resulted_in_click |
AggregatedQueryCount | A rolled-up, time-bucketed count for one query string, produced by the offline batch job (O) | query_text, count, time_bucket, decay_weighted_score |
TrieNode (serialized) | One node of the precomputed structure, representing one prefix — the actual unit stored and served by a shard (M, N) | prefix, children (pointers/offsets), top_k_completions (sorted query_text + score pairs) |
TrendingDelta | A short-lived, fast-refreshing overlay entry tracking a prefix's recent-activity spike, layered on top of the stale bulk trie (O) | prefix, top_trending_completions, window_start, window_end |
UserQueryHistory | A bounded, per-user record of recent searches/clicks used to personalize results (R) | user_id, recent_queries (small, capped list), last_updated |
I. API Design
-- client-facing:
GET /autocomplete?prefix={text}&limit=10 -> ranked list of { completion, score }
POST /events/click { user_id?, prefix, selected_completion } -> fire-and-forget feedback signal (R, O)
-- internal, cluster-facing, not client-facing:
(query log stream) every submitted search -> durable log (Kafka, Prerequisites G)
(batch aggregation job) periodic: query log -> counts + decay weighting -> rebuild trie top-K caches (O)
(trending aggregator) continuous: recent window of query log -> TrendingDelta per prefix (O)
(shard push) new trie version -> blue-green deploy onto serving shards (O, P)
Why GET /autocomplete is the only latency-critical endpoint in the whole system: everything else — logging a query, aggregating counts, rebuilding the trie, recording a click — happens off the request path entirely, on its own schedule (Patterns Playbook C). The one endpoint a user is actually waiting on synchronously is the read, and every other deep dive in this problem exists specifically to keep that one read cheap.
J. Basic / Naive High-Level Design (V1)
The simplest thing that satisfies the functional requirement in isolation: on every keystroke, run a live query against the query-log database — SELECT query_text, popularity FROM queries WHERE query_text LIKE 'prefix%' ORDER BY popularity DESC LIMIT 10 — and return whatever comes back.
(keystroke)"] --> API["Autocomplete API"] API --> DB[("Query Log DB
LIKE 'prefix%' + live ORDER BY")] DB --> API API --> Client
V1 — one live, sorted prefix scan against the database, recomputed from scratch on every single keystroke.
Why this breaks specifically on the traffic that matters most: for a long, specific prefix ("thermodynamics textbook re"), very few rows match, and the query is cheap regardless of how it's executed. But for exactly the short, common prefixes that make up a disproportionate share of real traffic (G) — "a", "th", "wea" — the LIKE 'prefix%' clause matches millions of rows, and the database has to scan and sort all of them, live, under a sub-100ms budget, on every one of thousands of concurrent keystrokes per second. The failure isn't a missing index — a leading, anchored prefix match can use one — the failure is recomputing an expensive rank-and-sort over an enormous, ever-growing candidate set from scratch, per request, for exactly the queries that arrive most often. An equally naive alternative — an in-memory trie with no precomputed ranking, which walks to the prefix node and then does a live DFS over its entire subtree to collect and sort every completion beneath it — has the identical flaw in a different shape: the subtree beneath a short, hot prefix is the largest subtree in the whole structure, so the traversal cost is again largest exactly where request volume is highest. Both point at the same fix: never rank live, at request time, over an unbounded candidate set — rank ahead of time, offline, and cache the small, already-sorted answer.
K. Data Model & Store Choice
The serving structure itself is a purpose-built, in-memory trie, not a general-purpose database: the access pattern is exclusively "walk down by character, read a small cached list at the end" — never a join, never a live aggregation — which is the same "match the store to the access pattern" reasoning from Prerequisites C and Distributed Cache's own storage choice (P11-K), just applied to a tree shape instead of a flat key space.
Raw query logs live in a high-write-throughput, append-only store (a durable log via Kafka, feeding into a wide-column or object store for the raw history) — this is by far the highest-volume write path in the system (billions of submitted searches, not counting the far larger keystroke volume that never reaches the log at all), and gets none of the read-optimized structure the trie has, because nothing ever queries it directly at request time.
Aggregated counts and the built trie version are the output of a batch job, stored as an immutable, versioned artifact (O) — pushed out to serving shards as a whole, rather than mutated in place, so a serving shard can swap to a new version atomically and cleanly roll back to the previous one if something's wrong with a rebuild.
Trending deltas and per-user history both live in Redis, TTL'd to their respective freshness window — both need fast, small, frequently-refreshed reads and writes, and both are inherently time-bounded, the same reasoning Notification System applies to its own short-lived digest batches (P12-K).
L. End-to-End Flow — A User Types 3 Characters and Gets Ranked Suggestions
The base, shared ranking is what the edge cache actually holds — the trending and personalization layers are merged in fresh on every request that reaches origin, never cached themselves at the edge.
M. Deep Dive 1 — Trie Data Structure for Prefix Search
Requirement this addresses: sub-100ms lookups (D, G) regardless of how many completions share a given prefix — and directly fixes V1's core flaw (J), which got slower exactly where traffic was heaviest.
Why a trie beats a sorted-list binary search: binary search over a single globally sorted array of query strings gets you to the start of the matching range in O(log n) — but you still then have to scan forward through however many entries share that prefix to know what's there, and the array itself needs contiguous, shiftable storage that makes inserts expensive. A trie sidesteps both problems: each node represents one character position, with children keyed by the next character, so walking from the root to the node representing a given prefix costs exactly O(prefix length) — a small, fixed number (bounded by the longest reasonable query, generally under 20-30 characters) — regardless of how large the overall corpus is or how many completions live beneath that node. The lookup cost is decoupled entirely from corpus size, which a sorted-array scan's forward-scan step is not.
Why it beats a live LIKE 'prefix%' scan: the same reasoning V1's callout already covered — a B-tree index on the query text can get an RDBMS to the start of a matching range about as efficiently as a trie gets to a node, but ranking and limiting the result still requires touching and sorting every matching row, live, per request. A trie's advantage isn't really about finding the prefix — it's that the node itself, once reached, can hold an already-ranked answer (N), which a raw index range scan has no equivalent way to do.
The memory cost of a naive trie, not fully addressed yet: a trie storing one node per character of every distinct query string, without any pruning, explodes in node count — a corpus of hundreds of millions of distinct strings (G) produces a proportionally enormous number of nodes, and a large fraction of that structure is long, rarely-visited tail branches (one-off typos, extremely specific long queries) that cost real memory to store and are almost never actually requested. This is exactly why the offline batch job (O) filters out low-frequency noise before building the served structure at all — the trie this system actually serves is built over the aggregated, filtered corpus, not the raw, unfiltered firehose of everything ever typed.
N. Deep Dive 2 — Precomputing Top-K at Each Trie Node
Requirement this addresses: the second half of V1's flaw (J) — reaching the right trie node quickly (M) is only half the problem; ranking whatever's beneath it still needs an answer.
Why a live subtree scan at request time is still broken, even with a trie: the naive move, once you have a trie, is to walk to the prefix's node and then DFS/BFS the entire subtree beneath it, collecting every completion, sorting by popularity, and taking the top K — live, on every request. This has the exact same shape of flaw as V1: the subtree beneath a short, common prefix ("a", "th") is the largest subtree in the entire structure, so this traversal is most expensive precisely for the requests that arrive most often, the same backwards-scaling problem in a new costume.
The fix: during the offline build (O), compute and cache the top-K completions — already fully sorted by score — directly at each node, not just at the leaves. A request is then answered by an O(prefix length) descent to the node, followed by an O(1) read of that node's small, pre-sorted list — no subtree traversal of any kind happens at request time, ever.
The cost, paid deliberately: the same popular completion ends up duplicated in the cached top-K list of every one of its ancestor nodes — "new york pizza" appears in the stored list for "n", "ne", "new", "new ", and so on. This is a direct trade of memory for read speed, the identical denormalize-for-reads move behind News Feed's precomputed FeedEntry store (P3-O), just applied to a tree's ancestor chain instead of a follower list.
The gap this creates: a node's cached top-K list is only as fresh as the last full rebuild — a query that starts spiking in popularity right now will not appear in any node's precomputed list until the next rebuild runs, which for a corpus this size is not instantaneous. This gap is exactly what O's real-time trending overlay exists to cover, deliberately, rather than trying to make the precomputed structure itself real-time.
O. Deep Dive 3 — Updating the Trie at Scale
Requirement this addresses: keeping the ranking reasonably fresh (D) without paying a write cost on the single hottest, most latency-sensitive path in the system.
Why real-time insert-per-query doesn't scale: the naive alternative to a periodic rebuild is to update the trie live — every submitted search walks from the root down to its own terminal node, bumping a counter at every ancestor along the way, and potentially re-sorting each of those ancestors' cached top-K lists (N) to reflect the new count. That's an O(query length) cascading write, fanning out to every ancestor node, triggered by the full query-submission volume (billions/day, G) — turning what should be a pure, cheap read path into a read path that also does a chain of writes on every request, which is precisely the coupling this whole design exists to avoid.
The offline batch pipeline, concretely: submitted queries stream into a durable, append-only log (Kafka, Prerequisites G); a periodic batch aggregation job (running hourly, or every few hours) counts query frequency over a rolling window, applies decay-weighting so recent activity counts more than stale activity, filters out low-frequency and long-tail noise (M), and rebuilds each affected node's top-K cache from that aggregated count. The new trie version is pushed out to serving shards as a whole and swapped in via a blue-green deploy — old version keeps serving until the new one is fully live, so a rebuild never produces a visible gap or inconsistency mid-swap.
The trending delta layer, filling the gap the batch job leaves open: a separate, much lighter-weight streaming aggregator tracks query counts over a short, recent window (minutes, not hours) — using a streaming framework or simply short-TTL'd Redis sorted sets keyed by prefix — and produces a small overlay of fast-rising completions per prefix. At read time (L), this overlay is merged alongside the stale-but-broad base trie result, so a genuinely breaking, fast-moving query can surface within minutes without ever making the underlying bulk structure itself real-time.
Why the staleness this accepts is fine, not a compromise: the base trie being minutes-to-hours behind the absolute latest activity costs nothing correctness-wise — the worst outcome is a slightly-less-optimal ranked suggestion, never a wrong answer a user could be harmed by. This trade-off is stated explicitly and generally in V.
P. Deep Dive 4 — Sharding the Trie
Requirement this addresses: the full corpus trie (G) doesn't fit comfortably in one machine's memory once node overhead and the top-K cache at every node is accounted for.
The fix — shard by prefix range, the tree's own natural ownership boundary: a top-level split by the first character (or first one to two characters) of the query assigns each shard server a whole subtree — one shard owns everything rooted under "a"–"f," another "g"–"m," and so on. This is the identical "shard by the boundary within which data is almost always accessed together" instinct named generally as Patterns Playbook Pattern 6, just applied to a tree's own structural branches instead of an entity ID like event_id or doc_id.
Routing, without needing a consistent-hashing ring at all: because the prefix characters themselves directly determine ownership — the first character alone tells you which shard to talk to — a thin routing layer, or even a small client-side static range map (mirroring Distributed Cache's client-resolves-shard approach, P11-I), can route a request without a full hash ring. The "key" here is structurally, not arbitrarily, distributed across the keyspace, so a simpler mechanism than consistent hashing suffices.
Rebalancing for uneven load: query volume is not evenly spread across the alphabet — some starting characters are searched far more often than others — so a naive equal split by letter produces uneven shard load. The fix is splitting further down the tree specifically for hot ranges (a heavily-searched letter gets its own dedicated shard, or even splits further into two-character prefixes, while several rarely-searched letters share one shard), decided from observed traffic statistics rather than assumed evenly at design time.
Q. Deep Dive 5 — Caching Hot Prefixes
Requirement this addresses: the extreme, structural skew toward short, common prefixes (D, G) — the same read fan-in shape as a viral post (P3-P) or a hot key in a distributed cache (P11-P), just triggered here by prefix length rather than by one specific piece of content.
Edge/CDN caching for the base ranking: the non-personalized response for a given (prefix, locale/region) is largely identical across huge numbers of users, and only changes as often as the underlying trie rebuilds (O) — minutes to hours. That makes it an unusually good fit for edge caching (Prerequisites H) with a short TTL matched to the trending-layer refresh interval: a request for "a" or "the" is, in the common case, served entirely from a nearby edge node and never reaches an origin trie shard at all.
A layered caching hierarchy, each layer catching what the one above it can't: edge/CDN (coarsest, longest-lived, region-shared) → a shared Redis or in-memory cache sitting in front of the origin trie shards for whatever does reach them (Prerequisites D; the same cache-aside layering Distributed Cache itself is built around, P11) → the trie shard itself, as the final source of truth. Each successive layer absorbs progressively more specific or less-cached traffic than the layer above it could serve.
Personalized and trending results deliberately bypass the edge layer: both vary per-user or refresh too frequently for a long-lived, shared edge cache to be worth the complexity — only the base, global ranking is ever edge-cached; the trending overlay (O) and personalization (R) are merged in fresh, at origin, after the cacheable base result has already been fetched.
R. Deep Dive 6 — Personalization
Requirement this addresses: blending a user's own search history with global popularity (D, E) without undermining the one thing that makes this design fast and cheap — a single, shared, precomputed structure.
Where personalization fits, and deliberately nowhere else: the base trie lookup (M, N) and the trending overlay (O) both stay entirely global and shareable exactly as designed. Personalization is applied as a final, per-request re-ranking step at the API/serving layer, never baked into the shared cached structure itself. A small, bounded, per-user history store (UserQueryHistory, H) — the user's own recent searches and clicks, keyed by user_id — is checked in parallel with the trie lookup; any of the user's own recent queries matching the current prefix are boosted toward the top or injected alongside the global top-K, and the two results are merged only after both have independently returned.
Why this ordering is the load-bearing decision, not an implementation detail: baking personalization directly into the trie — a separate per-user trie, say — would multiply G's already-substantial memory estimate by the number of users, which is obviously infeasible at any real scale. Keeping personalization as a thin overlay merged at read time preserves the one shared, cacheable base structure (which is what Q's edge caching actually depends on) while still letting an individual user's own signal genuinely matter.
Deliberately bounded scope: only a small, capped set of a user's most recent queries needs to be checked per request — not their entire lifetime history — keeping this lookup itself cheap and bounded, the same "keep the addition cheap enough not to threaten the fast path" discipline behind N's node-local top-K design.
S. Deep Dive 7 — Handling Typos / Fuzzy Matching
Requirement this addresses: real human typing includes real typos (D, E) — and an exact-prefix trie, by construction, only ever matches completions that literally begin with the exact characters typed so far; a single dropped or swapped letter ("pyton" for "python") walks the trie down a path with no useful completions beneath it at all.
Why the trie itself structurally cannot solve this: the entire reason a trie's lookup is fast (M) is that it only ever needs to follow the one exact path implied by the characters typed so far. Tolerating an edit distance of even one character means potentially considering many divergent paths at once — insertions, deletions, substitutions at every position — which defeats the O(prefix length) lookup cost that's the whole point of the structure in the first place. This isn't a missing feature to bolt on; it's a fundamentally different access pattern than what a trie is built for.
Lightweight mitigations, roughly in order of increasing cost:
- Phonetic/normalized secondary keys (a Soundex- or metaphone-style encoding) — index common queries under a phonetic key alongside the exact-text trie, so a phonetically similar misspelling can still be routed to the correctly-spelled term's bucket; cheap to add, but coarse and language-specific.
- A secondary fuzzy index over only the head of the popularity distribution — a small edit-distance-aware structure (a BK-tree, or an n-gram index) built over just the tens of thousands of most popular queries, not the full corpus; full fuzzy search over hundreds of millions of strings is expensive, but restricting it to the queries actually worth catching typos for keeps it tractable.
- A cheap client-side correction pass before the request is even sent — a small, bundled dictionary of extremely common misspellings, correcting the obvious case before it ever reaches the trie at all.
Worth naming explicitly, not glossing over: none of these are a complete, general-purpose fuzzy-search solution — this is an area where the honest answer is "good enough, bounded-cost mitigation for the common case," not "solve typo tolerance perfectly," and saying so directly is a stronger interview answer than quietly overpromising.
T. Full Solution at Scale — Final Architecture
(base ranking only, Q)"] EdgeCDN --> API["Autocomplete API"] API --> Router["Shard Router
(prefix-range map, P)"] Router --> ShardA["Trie Shard: a-f
(precomputed top-K, M, N)"] Router --> ShardB["Trie Shard: g-m"] Router --> ShardC["Trie Shard: n-z"] API --> DeltaCache[("Trending Delta Cache
Redis, short TTL, O")] API --> PersSvc["Personalization Service
(UserQueryHistory, R)"] API --> FuzzyIdx["Secondary Fuzzy Index
(head-of-distribution only, S)"] API --> Merge["Merge: base + trending + personalization"] Merge --> EdgeCDN Merge --> User subgraph OFFLINE [" Offline / Batch Pipeline (O) "] direction LR Logs[("Query Log Stream
Kafka, Prerequisites G")] Agg["Batch Aggregation Job
(counts + decay weighting)"] Build["Trie Builder
(rebuild top-K per node)"] Logs --> Agg --> Build end Build -.->|"blue-green version push"| ShardA Build -.->|"blue-green version push"| ShardB Build -.->|"blue-green version push"| ShardC subgraph STREAM [" Real-Time Trending Pipeline (O) "] direction LR RTLogs[("Recent Query Window")] Trend["Trending Aggregator"] RTLogs --> Trend end Trend --> DeltaCache
Everything above is the composition of M through S — the read path (top) stays cheap because every expensive computation (bottom) happens off of it entirely, on its own schedule.
U. Design Evolution
| Version | What Changed | Why |
|---|---|---|
| V1 | Live LIKE 'prefix%' scan with a live ORDER BY, recomputed on every keystroke (J) | Simplest possible thing that satisfies the functional requirement in isolation |
| V2 | An in-memory trie replaces the database scan for the lookup itself, but still does a live subtree DFS/sort per request | Fixes the "finding the prefix" half of V1's cost, but not the "ranking what's beneath it" half (M) |
| V3 | Top-K completions precomputed and cached directly at each trie node | Eliminates the live subtree traversal entirely — a request becomes a tree descent plus an O(1) read (N) |
| V4 | A defined offline batch pipeline (log → aggregate → rebuild) formalizes how and when V3's cached lists get refreshed | Replaces "however the trie happened to get built" with a repeatable, decay-weighted, noise-filtered process (O) |
| V5 | A real-time trending delta overlay is merged on top of the otherwise-stale bulk trie | Closes the freshness gap V4's periodic rebuild leaves open, without making the base structure itself real-time (O) |
| V6 | The trie is sharded by prefix range across multiple servers | The full corpus trie no longer fits one machine's memory once the corpus grows large enough (P) |
| V7 | Edge/CDN and shared-cache layers added in front of the origin shards | Absorbs the disproportionate share of traffic hitting a small number of short, hot prefixes (Q) |
| V8 | A personalization overlay merged at read time, on top of the shared base result | Lets a user's own history matter without duplicating the entire structure per user (R) |
| V9 | A secondary phonetic/fuzzy index added for the head of the popularity distribution | Mitigates the exact-prefix trie's structural inability to tolerate typos, for the queries most worth catching them on (S) |
What stays stable throughout: the small GET /autocomplete contract from I — every later refinement changes how the answer is computed, cached, or merged, never the basic shape of what a client asks for.
V. Consistency Model
This system is, unusually among the problems in this document, eventually consistent everywhere, with no narrow strong-consistency island at all — worth noticing as a genuine contrast to the more common shape named in Patterns Playbook J, where most designs default to eventual consistency but still carve out one small piece of state that must be strongly consistent (a seat's hold status, a leader lock). There is no equivalent here: nothing in this system is a scarce resource being contended over, and no single request's correctness depends on reading the absolute latest write. The base trie (N, O) can lag reality by minutes to hours; the trending overlay (O) lags by a smaller but still nonzero window; personalization (R) reads a user's own recent history, which itself may be a request or two behind the very latest click. In every one of these cases, the worst possible outcome of staleness is a suggestion list that's slightly less optimally ranked than it could be — never a lost booking, a double charge, or a duplicated message, the kinds of failures strong-consistency islands exist elsewhere in this document specifically to prevent. Because nothing here is actually correctness-critical in that sense, this design leans as far toward "stale but fast and cheap" as the read-heavy, staleness-tolerant requirement (D) allows, without ever needing to walk that trade-off back for a specific field.
W. Failure Scenarios
- A trie shard becomes unavailable: the shard router (P) detects the failure via health checks and routes requests for that prefix range to a replica of the same shard; if no replica is available, the system degrades to returning fewer or no results for that specific prefix range rather than failing the whole request path.
- A batch rebuild job fails partway through or produces a bad build: serving shards continue serving the last known-good trie version (O) — the blue-green swap only happens once a rebuild completes and validates successfully, so a broken rebuild simply never gets promoted, and the failure is visible to on-call rather than silently corrupting live results.
- The real-time trending aggregator falls behind or goes down: the trending overlay (O) is simply empty or stale for that window; the base trie (N) continues serving fine on its own, so a trending-pipeline outage degrades freshness for fast-moving queries specifically, without affecting the majority of traffic at all.
- The personalization service is down: the API merges the base and trending results without a personalization boost — every user sees the same shared, unpersonalized ranking, a graceful degradation rather than a failed request.
- A single hot prefix's edge cache entry expires simultaneously across many edge nodes: the same thundering-herd shape named for Distributed Cache's hot-key problem (P11-Q) — mitigated the same way, via request coalescing at origin so only one in-flight fetch per prefix reaches the shared cache/trie shard while concurrent misses wait on it.
- A malicious or buggy client hammers the same rare, expensive prefix directly, bypassing normal traffic patterns: the layered cache (Q) and standard per-client rate limiting (reused as a cross-cutting concern, Patterns Playbook K) bound the damage to that one client's own allotment, rather than letting it degrade the shared origin tier for everyone.
X. Bottlenecks & Scaling Boundary
| Stage | Actual Bottleneck | Fix Applied |
|---|---|---|
| V1 live prefix scan | Ranking recomputed live, per request, over an unbounded and largest-for-the-hottest-prefixes candidate set | Precomputed, cached top-K at each trie node (N) |
| Real-time insert-per-query | Every submitted search would cascade a write up every ancestor node, at full query-volume write rate, on the read-critical path | Offline batch rebuild plus a separate, lightweight real-time trending overlay (O) |
| Single-machine trie | The full corpus trie (G) exceeds one machine's safe memory budget once top-K caches are included | Sharding by prefix range across multiple servers (P) |
| Disproportionate load on short, common prefixes | A small set of prefixes accounts for a wildly outsized share of total request volume, regardless of how well the rest of the keyspace is distributed | Layered edge/CDN and shared caching in front of origin shards (Q) |
| Personalization baked into the shared structure | A per-user trie would multiply memory cost by the number of users — infeasible at any real scale | Personalization kept as a thin, bounded overlay merged at read time (R) |
| Exact-prefix-only matching | A single typo produces a dead-end path with no useful completions, for a real and common class of input | A bounded secondary fuzzy/phonetic index over only the most popular queries (S) |
Y. Design Trade-Offs
| Decision | Option A | Option B | Trade-off |
|---|---|---|---|
| Ranking computation | Live, per-request scan/sort | Precomputed top-K per trie node (chosen) | Live computation is simpler and always perfectly fresh, but scales worst exactly where traffic is heaviest; precomputing trades some staleness for making the hottest requests the cheapest ones |
| Freshness mechanism | Real-time insert-per-query | Periodic batch rebuild + real-time trending overlay (chosen) | Real-time insert keeps the structure always current but adds a cascading write to the hottest read path; the batch-plus-overlay split keeps the read path pure while still surfacing fast-moving trends within minutes |
| Trie placement | Single machine, unsharded | Sharded by prefix range (chosen) | Unsharded is simpler to operate but caps out once the corpus exceeds one machine's memory; sharding costs a routing layer in exchange for effectively unbounded horizontal scale |
| Personalization | Baked into a per-user structure | Thin overlay merged at read time (chosen) | A per-user structure gives the cleanest personalization but multiplies memory cost by user count; a merged overlay keeps one shared, cacheable base structure at the cost of a second, smaller lookup per request |
| Typo tolerance | None (exact-prefix only, chosen default) | Full fuzzy/edit-distance index over the entire corpus | Exact-prefix is fast and simple but misses a real class of common typing errors; full fuzzy search over the whole corpus would meaningfully raise both cost and lookup latency for a benefit concentrated in a minority of requests |
Z. Common Bad Designs
- A live
LIKE 'prefix%'database scan with a liveORDER BYon every keystroke. Scales worst on exactly the short, common prefixes that make up the majority of real traffic — the single most important mistake to avoid on this problem. - An in-memory trie with no precomputed ranking, doing a live subtree scan per request. Fixes the wrong half of the problem — finding the prefix fast doesn't help if ranking what's beneath it is still done live, and the subtree is largest exactly where traffic is heaviest.
- Inserting and re-ranking on every submitted query, live, against the serving structure. Turns a pure read path into a read-plus-cascading-write path at full query volume — the single change most likely to sink an otherwise-correct design under real load.
- One giant, unsharded trie assumed to fit on one machine regardless of corpus size. Works fine in a small-scale demo and falls over the moment the corpus estimate in G is taken seriously.
- Baking personalization directly into the shared, cached structure. Multiplies memory cost by the number of users and destroys the one thing (a single shared, cacheable base result) that makes edge caching viable at all.
- Ignoring staleness as a design constraint entirely, insisting every suggestion be perfectly real-time. Solves a problem this system doesn't actually have — the honest answer is that a few minutes of lag costs nothing here, and insisting otherwise adds real complexity for no real benefit.
AA. Interviewer Follow-Ups
| Follow-Up | Design Change | Components Changed | Components Unchanged |
|---|---|---|---|
| "How would you support this across multiple languages/locales?" | A separate trie (or a separate top-level branch) per language/locale, with routing decided by the client's declared locale before the prefix-range shard lookup ever happens | Sharding/routing layer, offline build pipeline (now per-locale) | The trie structure itself, precomputed top-K, trending overlay mechanism |
| "What about location-aware autocomplete, like a maps search box?" | Blend the text-prefix trie with a geospatial signal — either a geohash-prefixed branch of the structure or a separate geospatial index merged in at read time, reusing Ride-Sharing's geospatial primer reasoning (Prerequisites N) rather than inventing a new spatial mechanism | Merge/ranking step, a new geospatial signal source | Core trie structure, batch rebuild pipeline, edge caching |
| "How do you keep offensive or private (PII-containing) queries from surfacing as suggestions?" | A filtering/blocklist step added inside the offline aggregation job (O), before any query ever contributes to a node's top-K cache — kept upstream of the trie build, not applied as a per-request check | The batch aggregation job only | Everything downstream — the trie, sharding, caching, personalization all stay identical |
| "What if this needs to autocomplete structured entities — products, not just free-text queries?" | AggregatedQueryCount (H) gains associated entity metadata (inventory, price, category), and ranking blends textual popularity with entity-specific signals — a genuine extension to the ranking function, not a structural rewrite | Core entities, the ranking/scoring function used during the batch build | The trie structure, sharding, and caching layers themselves |
AB. Interviewer Evaluation Rubric
- Weak candidate: proposes a live database query or an unranked in-memory list with no precomputation at all, and has no answer for why a short, common prefix would behave any differently from a long, rare one under load.
- Mid-level: correctly reaches for a trie and recognizes it needs precomputed ranking, but treats "rebuild the trie" as a hand-wave rather than a concrete pipeline, and hasn't considered what happens once the corpus no longer fits on one machine.
- Senior: proactively separates the slow-changing bulk structure from a fast-moving trending overlay; reasons concretely about sharding by prefix range rather than defaulting to a generic consistent-hashing ring without asking whether the keyspace's structure already gives a simpler routing answer; is explicit that this system has no strong-consistency requirement anywhere and says why.
- Staff+: raises the hot-prefix caching problem unprompted and connects it explicitly to the same shape as a hot key or viral content elsewhere; is honest about typo tolerance being a bounded, cost-conscious mitigation rather than a fully solved problem; reasons about where personalization can and can't be added without destroying the shared-cache economics the whole design depends on.
AC. Interview Simulation
AD. Online Solution Comparison
| Source | Approach | Strength | Weakness | Worth Learning |
|---|---|---|---|---|
| Elasticsearch Completion Suggester (official docs) | Builds a Finite State Transducer (FST) — a compressed, trie-like structure — at index time from configured suggestion inputs and weights, held entirely in memory for extremely fast prefix lookups; natively supports fuzzy matching via a configurable Damerau-Levenshtein edit-distance parameter | A production-grade, real confirmation that a precomputed, in-memory, trie-shaped structure built ahead of time (not queried live) is the standard real-world answer to this exact problem — and that fuzzy tolerance (S) is commonly bolted on as a configurable parameter rather than solved from scratch | Rebuilding the FST means reindexing the underlying suggester data — genuinely real-time updates aren't native, which is exactly why this design layers a separate trending overlay (O) rather than expecting the base structure to be live | Confirms that "precompute a prefix-optimized structure offline, keep it in memory, tolerate rebuild lag" is the real production pattern, not an interview simplification |
| Google Search "About autocomplete predictions" (official help documentation) | Documents that predictions are generated from real searches performed by users, are explicitly influenced by trending topics and rapidly updated for breaking events, and are personalized using signals including a signed-in user's own search history and general location | A direct, official confirmation that a real, massive-scale autocomplete system separates "broad historical popularity" from "trending right now" and layers personalization on top of both — the exact three-way split this design's O and R sections are built around | The documentation intentionally doesn't disclose ranking internals or infrastructure — useful for confirming the shape of the problem, not for replicating an implementation | Confirms trending-as-a-distinct-signal and personalization-as-an-overlay are real, named, production concerns at the largest possible scale, not interview embellishments |
| "Auto Complete with Redis" (Salvatore Sanfilippo / antirez, Redis creator's blog) | Describes simulating a trie using a single Redis sorted set, storing every prefix of every term as a member and using range queries (ZRANGEBYLEX-style lexicographic range scans) to retrieve completions for a given prefix, avoiding the need to build and maintain an actual pointer-based tree structure in application memory | A real, widely-referenced open-source pattern for implementing trie-like prefix search using an off-the-shelf data store rather than custom infrastructure — a legitimate, lighter-weight alternative to a bespoke trie server for smaller-scale versions of this problem | Storing every prefix of every term multiplies storage several-fold over a true trie's shared-prefix structure, and this approach doesn't natively give you a precomputed, ranked top-K per node (N) — ranking still has to be layered on separately | A concrete, credible demonstration that the core "prefix range lookup" idea behind a trie can be implemented multiple different ways depending on scale — worth knowing this is a spectrum, not one single canonical implementation |
AE. Interview Cheat Sheet
| Problem | Search Autocomplete / Typeahead |
| Difficulty | Intermediate → Advanced (simple read path, genuinely hard freshness/sharding/caching surface underneath it) |
| Core requirements | Ranked prefix completion, sub-100ms per keystroke, updates over time as trends shift, tolerant of scale-driven staleness |
| Central decision | Trie with precomputed top-K per node, rebuilt offline from aggregated logs, plus a real-time trending overlay, sharded by prefix range, cached aggressively at the edge for hot prefixes |
| Why not live ranking | A live scan/sort or a live subtree traversal is slowest exactly on the short, common prefixes that account for most real traffic — precomputing inverts that relationship |
| Freshness split | Two distinct mechanisms: a slow, broad, decay-weighted batch rebuild for the bulk ranking; a fast, narrow streaming overlay for genuinely trending queries |
| Sharding | By prefix range — the tree's own structural boundary supplies the shard key directly, no consistent-hashing ring required |
| Caching | Layered edge/CDN → shared cache → origin shard, but only for the shared, non-personalized base ranking |
| Personalization & typos | Both are thin, bounded overlays merged at read time — never baked into the shared precomputed structure itself |
| Consistency | Eventually consistent everywhere, with no strong-consistency island at all — staleness here costs ranking quality, never correctness |
| Common mistakes | Live prefix scans/sorts; a trie with no precomputed ranking; real-time insert-per-query into the serving structure; one unsharded trie; personalization baked into the shared cache; expecting perfect typo tolerance for free |
| What NOT to over-engineer | A fully general fuzzy-search engine and a from-scratch ML ranking model — name both as real, bounded extensions (S, AA) rather than designing them from first principles in the room |