13. Search Autocomplete / Typeahead Intermediate → Advanced

A. Realistic Interview Prompt

"Design a search autocomplete / typeahead system — the kind of thing sitting behind the Google or Amazon search box — that suggests likely completions to a user as they type, ranked by how popular or relevant each completion actually is, and fast enough that it never feels like it's lagging behind the keyboard."

Every character a user types is, underneath, a fresh request for "rank and return the best few completions for this exact prefix," fired at keystroke speed, at a volume that dwarfs the number of searches actually submitted — a single search is preceded by several throwaway prefix requests that never turn into anything. The honest, load-bearing answer this problem is actually testing is one integrated decision: a Trie (prefix tree) with the top-K completions precomputed and cached at each node, rebuilt periodically offline from aggregated query logs, with a fast-moving "trending" overlay merged on top of that otherwise-stale structure, sharded by prefix range once the corpus outgrows one machine, and aggressively cached at the edge for the small set of short, extremely hot prefixes that account for a disproportionate share of all traffic.

B. Difficulty

Intermediate, growing into genuine distributed-systems territory in the deep dives — the read path itself (walk a tree, return a cached list) is conceptually simple once you know to reach for a trie at all; what makes this problem hard is everything around that read: keeping the structure fresh without paying a write cost on every keystroke (Patterns Playbook C), sharding a tree-shaped data structure rather than a flat keyspace (Patterns Playbook G), and layering personalization and typo-tolerance on top without breaking the one thing that makes the whole design fast — a shared, cacheable, precomputed structure.

C. What This Problem Tests

D. Requirements

Functional

  1. Given a partial search string (a prefix), return a ranked list of the top-K most likely completions
  2. Ranking reflects overall popularity across all users — how often a completion has actually been searched — not simple alphabetical order
  3. Suggestions update, on some reasonable delay, to reflect genuinely new or newly-trending queries, not just what was popular when the system was first built
  4. Results are returned per keystroke, incrementally, as the user's prefix grows or shrinks
  5. (Extension, addressed in R/S) results can be personalized per user and tolerant of minor typos

Non-Functional

Out of scope: full-text search over document contents (this system only autocompletes search-box query strings, not searching within results); a complete natural-language query-understanding layer; building a production-grade spelling-correction model from scratch (S names the mitigations available without claiming to design one in the room).

E. Clarifying Questions

#QuestionWhy It MattersDesign Decision Affected
1Does ranking need to reflect real-time trending activity (breaking news, a sudden spike), or is "popular over the last several days/weeks" an acceptable basis for ranking?Determines whether a fast-moving real-time overlay is a first-class requirement or a later extensionDeep Dive 3 (O), Capacity Estimation (G)
2Should suggestions be personalized to an individual user's own search history, or is one shared, global ranking sufficient?Changes whether personalization is a core requirement shaping the read path from the start, or a bolt-on layerDeep Dive 6 (R)
3Do we need to tolerate typos/misspellings in the prefix itself, or is exact-prefix matching acceptable?An exact-prefix trie structurally cannot answer a typo-tolerant query on its own — this changes whether a secondary mechanism is required at allDeep Dive 7 (S)
4Roughly how large is the corpus of distinct query strings we're ranking over, and how long can completions realistically be?Directly drives whether the precomputed structure fits comfortably on one machine or requires sharding from the startCapacity Estimation (G), Deep Dive 4 (P)
5Is this general web-search autocomplete, or scoped to a narrower domain (product search, a company directory)?A narrower, smaller corpus may never need sharding at all — worth confirming before assuming the full-scale version is requiredAssumptions (F), Capacity Estimation (G)

F. Assumptions

G. Capacity Estimation

MetricValue / CalculationWhy It Matters
Total submitted searchesIllustratively, ~1B searches/day platform-wideThe baseline "real" query volume — everything else in this table is a multiple of this number, not a replacement for it
Autocomplete request volumeEach submitted search is preceded by roughly 5–7 keystroke-triggered prefix requests (after client-side debouncing) → ~6–7B autocomplete requests/day ≈ ~75,000/sec averageThis, not the search-submission rate, is the number the read path has to be sized for — the actual dominant traffic in this entire system
Peak traffic skewPeak hours plus the natural skew toward short/common prefixes push the effective rate on hot paths to ~4–5x average during peak windowsConfirms this is fundamentally a read-fan-in problem concentrated on a small number of very hot prefixes, not an evenly-spread load
Distinct query corpus after filteringIllustratively, ~100–200M frequently-seen distinct query strings survive the offline aggregation job's noise/long-tail filtering (out of a much larger raw logged corpus)Sets the outer bound on trie node count — everything below this line filters out the one-off, rarely-repeated tail that isn't worth precomputing ranking for at all
Trie memory footprint~200M shared-prefix nodes × ~500–600 bytes/node (children pointers + a small top-10 cached completion list per node) ≈ roughly 100–120 GB for the full structureComfortably beyond a single machine's safe in-memory budget — the concrete number that makes sharding (P) a necessity, not a hypothetical
Per-request latency budgetSub-100ms end to end, ideally well under that, to feel instantaneous relative to typing speedRules out anything that scans, joins, or re-ranks a large candidate set live per request — the entire justification for precomputing top-K (N) ahead of time

H. Core Entities

EntityPurposeKey Fields
QueryLogEntryOne raw record of a search actually submitted by a user, the ultimate source of all ranking signalquery_text, user_id (optional), timestamp, region/locale, resulted_in_click
AggregatedQueryCountA rolled-up, time-bucketed count for one query string, produced by the offline batch job (O)query_text, count, time_bucket, decay_weighted_score
TrieNode (serialized)One node of the precomputed structure, representing one prefix — the actual unit stored and served by a shard (M, N)prefix, children (pointers/offsets), top_k_completions (sorted query_text + score pairs)
TrendingDeltaA short-lived, fast-refreshing overlay entry tracking a prefix's recent-activity spike, layered on top of the stale bulk trie (O)prefix, top_trending_completions, window_start, window_end
UserQueryHistoryA bounded, per-user record of recent searches/clicks used to personalize results (R)user_id, recent_queries (small, capped list), last_updated

I. API Design

-- client-facing:
GET /autocomplete?prefix={text}&limit=10          -> ranked list of { completion, score }
POST /events/click { user_id?, prefix, selected_completion }  -> fire-and-forget feedback signal (R, O)
-- internal, cluster-facing, not client-facing:
(query log stream) every submitted search -> durable log (Kafka, Prerequisites G)
(batch aggregation job) periodic: query log -> counts + decay weighting -> rebuild trie top-K caches (O)
(trending aggregator) continuous: recent window of query log -> TrendingDelta per prefix (O)
(shard push) new trie version -> blue-green deploy onto serving shards (O, P)

Why GET /autocomplete is the only latency-critical endpoint in the whole system: everything else — logging a query, aggregating counts, rebuilding the trie, recording a click — happens off the request path entirely, on its own schedule (Patterns Playbook C). The one endpoint a user is actually waiting on synchronously is the read, and every other deep dive in this problem exists specifically to keep that one read cheap.

J. Basic / Naive High-Level Design (V1)

The simplest thing that satisfies the functional requirement in isolation: on every keystroke, run a live query against the query-log database — SELECT query_text, popularity FROM queries WHERE query_text LIKE 'prefix%' ORDER BY popularity DESC LIMIT 10 — and return whatever comes back.

flowchart LR Client["Browser
(keystroke)"] --> API["Autocomplete API"] API --> DB[("Query Log DB
LIKE 'prefix%' + live ORDER BY")] DB --> API API --> Client

V1 — one live, sorted prefix scan against the database, recomputed from scratch on every single keystroke.

Why this breaks specifically on the traffic that matters most: for a long, specific prefix ("thermodynamics textbook re"), very few rows match, and the query is cheap regardless of how it's executed. But for exactly the short, common prefixes that make up a disproportionate share of real traffic (G) — "a", "th", "wea" — the LIKE 'prefix%' clause matches millions of rows, and the database has to scan and sort all of them, live, under a sub-100ms budget, on every one of thousands of concurrent keystrokes per second. The failure isn't a missing index — a leading, anchored prefix match can use one — the failure is recomputing an expensive rank-and-sort over an enormous, ever-growing candidate set from scratch, per request, for exactly the queries that arrive most often. An equally naive alternative — an in-memory trie with no precomputed ranking, which walks to the prefix node and then does a live DFS over its entire subtree to collect and sort every completion beneath it — has the identical flaw in a different shape: the subtree beneath a short, hot prefix is the largest subtree in the whole structure, so the traversal cost is again largest exactly where request volume is highest. Both point at the same fix: never rank live, at request time, over an unbounded candidate set — rank ahead of time, offline, and cache the small, already-sorted answer.

K. Data Model & Store Choice

The serving structure itself is a purpose-built, in-memory trie, not a general-purpose database: the access pattern is exclusively "walk down by character, read a small cached list at the end" — never a join, never a live aggregation — which is the same "match the store to the access pattern" reasoning from Prerequisites C and Distributed Cache's own storage choice (P11-K), just applied to a tree shape instead of a flat key space.

Raw query logs live in a high-write-throughput, append-only store (a durable log via Kafka, feeding into a wide-column or object store for the raw history) — this is by far the highest-volume write path in the system (billions of submitted searches, not counting the far larger keystroke volume that never reaches the log at all), and gets none of the read-optimized structure the trie has, because nothing ever queries it directly at request time.

Aggregated counts and the built trie version are the output of a batch job, stored as an immutable, versioned artifact (O) — pushed out to serving shards as a whole, rather than mutated in place, so a serving shard can swap to a new version atomically and cleanly roll back to the previous one if something's wrong with a rebuild.

Trending deltas and per-user history both live in Redis, TTL'd to their respective freshness window — both need fast, small, frequently-refreshed reads and writes, and both are inherently time-bounded, the same reasoning Notification System applies to its own short-lived digest batches (P12-K).

L. End-to-End Flow — A User Types 3 Characters and Gets Ranked Suggestions

sequenceDiagram participant U as User (typing) participant Edge as Edge/CDN Cache (Q) participant API as Autocomplete API participant Router as Shard Router (P) participant Shard as Trie Shard (M, N) participant Delta as Trending Delta Cache (O) participant Pers as Personalization Service (R) U->>Edge: GET /autocomplete?prefix=app alt hot prefix, cached at edge Edge-->>U: cached base ranking, returned directly else miss, falls through to origin Edge->>API: forward request API->>Router: which shard owns "app"? (P) Router-->>API: shard address API->>Shard: walk trie to "app" node, read cached top-K (N) Shard-->>API: base ranked list API->>Delta: any trending completions under "app"? (O) Delta-->>API: trending overlay (may be empty) API->>Pers: user's own recent queries matching "app"? (R) Pers-->>API: personalized boosts (may be empty) API->>API: merge base + trending + personalization API-->>Edge: response (cached at edge for the base portion) Edge-->>U: final ranked suggestions end

The base, shared ranking is what the edge cache actually holds — the trending and personalization layers are merged in fresh on every request that reaches origin, never cached themselves at the edge.

M. Deep Dive 1 — Trie Data Structure for Prefix Search

Requirement this addresses: sub-100ms lookups (D, G) regardless of how many completions share a given prefix — and directly fixes V1's core flaw (J), which got slower exactly where traffic was heaviest.

Why a trie beats a sorted-list binary search: binary search over a single globally sorted array of query strings gets you to the start of the matching range in O(log n) — but you still then have to scan forward through however many entries share that prefix to know what's there, and the array itself needs contiguous, shiftable storage that makes inserts expensive. A trie sidesteps both problems: each node represents one character position, with children keyed by the next character, so walking from the root to the node representing a given prefix costs exactly O(prefix length) — a small, fixed number (bounded by the longest reasonable query, generally under 20-30 characters) — regardless of how large the overall corpus is or how many completions live beneath that node. The lookup cost is decoupled entirely from corpus size, which a sorted-array scan's forward-scan step is not.

Why it beats a live LIKE 'prefix%' scan: the same reasoning V1's callout already covered — a B-tree index on the query text can get an RDBMS to the start of a matching range about as efficiently as a trie gets to a node, but ranking and limiting the result still requires touching and sorting every matching row, live, per request. A trie's advantage isn't really about finding the prefix — it's that the node itself, once reached, can hold an already-ranked answer (N), which a raw index range scan has no equivalent way to do.

The memory cost of a naive trie, not fully addressed yet: a trie storing one node per character of every distinct query string, without any pruning, explodes in node count — a corpus of hundreds of millions of distinct strings (G) produces a proportionally enormous number of nodes, and a large fraction of that structure is long, rarely-visited tail branches (one-off typos, extremely specific long queries) that cost real memory to store and are almost never actually requested. This is exactly why the offline batch job (O) filters out low-frequency noise before building the served structure at all — the trie this system actually serves is built over the aggregated, filtered corpus, not the raw, unfiltered firehose of everything ever typed.

N. Deep Dive 2 — Precomputing Top-K at Each Trie Node

Requirement this addresses: the second half of V1's flaw (J) — reaching the right trie node quickly (M) is only half the problem; ranking whatever's beneath it still needs an answer.

Why a live subtree scan at request time is still broken, even with a trie: the naive move, once you have a trie, is to walk to the prefix's node and then DFS/BFS the entire subtree beneath it, collecting every completion, sorting by popularity, and taking the top K — live, on every request. This has the exact same shape of flaw as V1: the subtree beneath a short, common prefix ("a", "th") is the largest subtree in the entire structure, so this traversal is most expensive precisely for the requests that arrive most often, the same backwards-scaling problem in a new costume.

The fix: during the offline build (O), compute and cache the top-K completions — already fully sorted by score — directly at each node, not just at the leaves. A request is then answered by an O(prefix length) descent to the node, followed by an O(1) read of that node's small, pre-sorted list — no subtree traversal of any kind happens at request time, ever.

The cost, paid deliberately: the same popular completion ends up duplicated in the cached top-K list of every one of its ancestor nodes — "new york pizza" appears in the stored list for "n", "ne", "new", "new ", and so on. This is a direct trade of memory for read speed, the identical denormalize-for-reads move behind News Feed's precomputed FeedEntry store (P3-O), just applied to a tree's ancestor chain instead of a follower list.

The gap this creates: a node's cached top-K list is only as fresh as the last full rebuild — a query that starts spiking in popularity right now will not appear in any node's precomputed list until the next rebuild runs, which for a corpus this size is not instantaneous. This gap is exactly what O's real-time trending overlay exists to cover, deliberately, rather than trying to make the precomputed structure itself real-time.

O. Deep Dive 3 — Updating the Trie at Scale

Requirement this addresses: keeping the ranking reasonably fresh (D) without paying a write cost on the single hottest, most latency-sensitive path in the system.

Why real-time insert-per-query doesn't scale: the naive alternative to a periodic rebuild is to update the trie live — every submitted search walks from the root down to its own terminal node, bumping a counter at every ancestor along the way, and potentially re-sorting each of those ancestors' cached top-K lists (N) to reflect the new count. That's an O(query length) cascading write, fanning out to every ancestor node, triggered by the full query-submission volume (billions/day, G) — turning what should be a pure, cheap read path into a read path that also does a chain of writes on every request, which is precisely the coupling this whole design exists to avoid.

The offline batch pipeline, concretely: submitted queries stream into a durable, append-only log (Kafka, Prerequisites G); a periodic batch aggregation job (running hourly, or every few hours) counts query frequency over a rolling window, applies decay-weighting so recent activity counts more than stale activity, filters out low-frequency and long-tail noise (M), and rebuilds each affected node's top-K cache from that aggregated count. The new trie version is pushed out to serving shards as a whole and swapped in via a blue-green deploy — old version keeps serving until the new one is fully live, so a rebuild never produces a visible gap or inconsistency mid-swap.

The trending delta layer, filling the gap the batch job leaves open: a separate, much lighter-weight streaming aggregator tracks query counts over a short, recent window (minutes, not hours) — using a streaming framework or simply short-TTL'd Redis sorted sets keyed by prefix — and produces a small overlay of fast-rising completions per prefix. At read time (L), this overlay is merged alongside the stale-but-broad base trie result, so a genuinely breaking, fast-moving query can surface within minutes without ever making the underlying bulk structure itself real-time.

Why the staleness this accepts is fine, not a compromise: the base trie being minutes-to-hours behind the absolute latest activity costs nothing correctness-wise — the worst outcome is a slightly-less-optimal ranked suggestion, never a wrong answer a user could be harmed by. This trade-off is stated explicitly and generally in V.

P. Deep Dive 4 — Sharding the Trie

Requirement this addresses: the full corpus trie (G) doesn't fit comfortably in one machine's memory once node overhead and the top-K cache at every node is accounted for.

The fix — shard by prefix range, the tree's own natural ownership boundary: a top-level split by the first character (or first one to two characters) of the query assigns each shard server a whole subtree — one shard owns everything rooted under "a"–"f," another "g"–"m," and so on. This is the identical "shard by the boundary within which data is almost always accessed together" instinct named generally as Patterns Playbook Pattern 6, just applied to a tree's own structural branches instead of an entity ID like event_id or doc_id.

Routing, without needing a consistent-hashing ring at all: because the prefix characters themselves directly determine ownership — the first character alone tells you which shard to talk to — a thin routing layer, or even a small client-side static range map (mirroring Distributed Cache's client-resolves-shard approach, P11-I), can route a request without a full hash ring. The "key" here is structurally, not arbitrarily, distributed across the keyspace, so a simpler mechanism than consistent hashing suffices.

Rebalancing for uneven load: query volume is not evenly spread across the alphabet — some starting characters are searched far more often than others — so a naive equal split by letter produces uneven shard load. The fix is splitting further down the tree specifically for hot ranges (a heavily-searched letter gets its own dedicated shard, or even splits further into two-character prefixes, while several rarely-searched letters share one shard), decided from observed traffic statistics rather than assumed evenly at design time.

Q. Deep Dive 5 — Caching Hot Prefixes

Requirement this addresses: the extreme, structural skew toward short, common prefixes (D, G) — the same read fan-in shape as a viral post (P3-P) or a hot key in a distributed cache (P11-P), just triggered here by prefix length rather than by one specific piece of content.

Edge/CDN caching for the base ranking: the non-personalized response for a given (prefix, locale/region) is largely identical across huge numbers of users, and only changes as often as the underlying trie rebuilds (O) — minutes to hours. That makes it an unusually good fit for edge caching (Prerequisites H) with a short TTL matched to the trending-layer refresh interval: a request for "a" or "the" is, in the common case, served entirely from a nearby edge node and never reaches an origin trie shard at all.

A layered caching hierarchy, each layer catching what the one above it can't: edge/CDN (coarsest, longest-lived, region-shared) → a shared Redis or in-memory cache sitting in front of the origin trie shards for whatever does reach them (Prerequisites D; the same cache-aside layering Distributed Cache itself is built around, P11) → the trie shard itself, as the final source of truth. Each successive layer absorbs progressively more specific or less-cached traffic than the layer above it could serve.

Personalized and trending results deliberately bypass the edge layer: both vary per-user or refresh too frequently for a long-lived, shared edge cache to be worth the complexity — only the base, global ranking is ever edge-cached; the trending overlay (O) and personalization (R) are merged in fresh, at origin, after the cacheable base result has already been fetched.

R. Deep Dive 6 — Personalization

Requirement this addresses: blending a user's own search history with global popularity (D, E) without undermining the one thing that makes this design fast and cheap — a single, shared, precomputed structure.

Where personalization fits, and deliberately nowhere else: the base trie lookup (M, N) and the trending overlay (O) both stay entirely global and shareable exactly as designed. Personalization is applied as a final, per-request re-ranking step at the API/serving layer, never baked into the shared cached structure itself. A small, bounded, per-user history store (UserQueryHistory, H) — the user's own recent searches and clicks, keyed by user_id — is checked in parallel with the trie lookup; any of the user's own recent queries matching the current prefix are boosted toward the top or injected alongside the global top-K, and the two results are merged only after both have independently returned.

Why this ordering is the load-bearing decision, not an implementation detail: baking personalization directly into the trie — a separate per-user trie, say — would multiply G's already-substantial memory estimate by the number of users, which is obviously infeasible at any real scale. Keeping personalization as a thin overlay merged at read time preserves the one shared, cacheable base structure (which is what Q's edge caching actually depends on) while still letting an individual user's own signal genuinely matter.

Deliberately bounded scope: only a small, capped set of a user's most recent queries needs to be checked per request — not their entire lifetime history — keeping this lookup itself cheap and bounded, the same "keep the addition cheap enough not to threaten the fast path" discipline behind N's node-local top-K design.

S. Deep Dive 7 — Handling Typos / Fuzzy Matching

Requirement this addresses: real human typing includes real typos (D, E) — and an exact-prefix trie, by construction, only ever matches completions that literally begin with the exact characters typed so far; a single dropped or swapped letter ("pyton" for "python") walks the trie down a path with no useful completions beneath it at all.

Why the trie itself structurally cannot solve this: the entire reason a trie's lookup is fast (M) is that it only ever needs to follow the one exact path implied by the characters typed so far. Tolerating an edit distance of even one character means potentially considering many divergent paths at once — insertions, deletions, substitutions at every position — which defeats the O(prefix length) lookup cost that's the whole point of the structure in the first place. This isn't a missing feature to bolt on; it's a fundamentally different access pattern than what a trie is built for.

Lightweight mitigations, roughly in order of increasing cost:

Worth naming explicitly, not glossing over: none of these are a complete, general-purpose fuzzy-search solution — this is an area where the honest answer is "good enough, bounded-cost mitigation for the common case," not "solve typo tolerance perfectly," and saying so directly is a stronger interview answer than quietly overpromising.

T. Full Solution at Scale — Final Architecture

flowchart TB User["User (typing)"] --> EdgeCDN["Edge / CDN Cache
(base ranking only, Q)"] EdgeCDN --> API["Autocomplete API"] API --> Router["Shard Router
(prefix-range map, P)"] Router --> ShardA["Trie Shard: a-f
(precomputed top-K, M, N)"] Router --> ShardB["Trie Shard: g-m"] Router --> ShardC["Trie Shard: n-z"] API --> DeltaCache[("Trending Delta Cache
Redis, short TTL, O")] API --> PersSvc["Personalization Service
(UserQueryHistory, R)"] API --> FuzzyIdx["Secondary Fuzzy Index
(head-of-distribution only, S)"] API --> Merge["Merge: base + trending + personalization"] Merge --> EdgeCDN Merge --> User subgraph OFFLINE [" Offline / Batch Pipeline (O) "] direction LR Logs[("Query Log Stream
Kafka, Prerequisites G")] Agg["Batch Aggregation Job
(counts + decay weighting)"] Build["Trie Builder
(rebuild top-K per node)"] Logs --> Agg --> Build end Build -.->|"blue-green version push"| ShardA Build -.->|"blue-green version push"| ShardB Build -.->|"blue-green version push"| ShardC subgraph STREAM [" Real-Time Trending Pipeline (O) "] direction LR RTLogs[("Recent Query Window")] Trend["Trending Aggregator"] RTLogs --> Trend end Trend --> DeltaCache

Everything above is the composition of M through S — the read path (top) stays cheap because every expensive computation (bottom) happens off of it entirely, on its own schedule.

U. Design Evolution

VersionWhat ChangedWhy
V1Live LIKE 'prefix%' scan with a live ORDER BY, recomputed on every keystroke (J)Simplest possible thing that satisfies the functional requirement in isolation
V2An in-memory trie replaces the database scan for the lookup itself, but still does a live subtree DFS/sort per requestFixes the "finding the prefix" half of V1's cost, but not the "ranking what's beneath it" half (M)
V3Top-K completions precomputed and cached directly at each trie nodeEliminates the live subtree traversal entirely — a request becomes a tree descent plus an O(1) read (N)
V4A defined offline batch pipeline (log → aggregate → rebuild) formalizes how and when V3's cached lists get refreshedReplaces "however the trie happened to get built" with a repeatable, decay-weighted, noise-filtered process (O)
V5A real-time trending delta overlay is merged on top of the otherwise-stale bulk trieCloses the freshness gap V4's periodic rebuild leaves open, without making the base structure itself real-time (O)
V6The trie is sharded by prefix range across multiple serversThe full corpus trie no longer fits one machine's memory once the corpus grows large enough (P)
V7Edge/CDN and shared-cache layers added in front of the origin shardsAbsorbs the disproportionate share of traffic hitting a small number of short, hot prefixes (Q)
V8A personalization overlay merged at read time, on top of the shared base resultLets a user's own history matter without duplicating the entire structure per user (R)
V9A secondary phonetic/fuzzy index added for the head of the popularity distributionMitigates the exact-prefix trie's structural inability to tolerate typos, for the queries most worth catching them on (S)

What stays stable throughout: the small GET /autocomplete contract from I — every later refinement changes how the answer is computed, cached, or merged, never the basic shape of what a client asks for.

V. Consistency Model

This system is, unusually among the problems in this document, eventually consistent everywhere, with no narrow strong-consistency island at all — worth noticing as a genuine contrast to the more common shape named in Patterns Playbook J, where most designs default to eventual consistency but still carve out one small piece of state that must be strongly consistent (a seat's hold status, a leader lock). There is no equivalent here: nothing in this system is a scarce resource being contended over, and no single request's correctness depends on reading the absolute latest write. The base trie (N, O) can lag reality by minutes to hours; the trending overlay (O) lags by a smaller but still nonzero window; personalization (R) reads a user's own recent history, which itself may be a request or two behind the very latest click. In every one of these cases, the worst possible outcome of staleness is a suggestion list that's slightly less optimally ranked than it could be — never a lost booking, a double charge, or a duplicated message, the kinds of failures strong-consistency islands exist elsewhere in this document specifically to prevent. Because nothing here is actually correctness-critical in that sense, this design leans as far toward "stale but fast and cheap" as the read-heavy, staleness-tolerant requirement (D) allows, without ever needing to walk that trade-off back for a specific field.

W. Failure Scenarios

X. Bottlenecks & Scaling Boundary

StageActual BottleneckFix Applied
V1 live prefix scanRanking recomputed live, per request, over an unbounded and largest-for-the-hottest-prefixes candidate setPrecomputed, cached top-K at each trie node (N)
Real-time insert-per-queryEvery submitted search would cascade a write up every ancestor node, at full query-volume write rate, on the read-critical pathOffline batch rebuild plus a separate, lightweight real-time trending overlay (O)
Single-machine trieThe full corpus trie (G) exceeds one machine's safe memory budget once top-K caches are includedSharding by prefix range across multiple servers (P)
Disproportionate load on short, common prefixesA small set of prefixes accounts for a wildly outsized share of total request volume, regardless of how well the rest of the keyspace is distributedLayered edge/CDN and shared caching in front of origin shards (Q)
Personalization baked into the shared structureA per-user trie would multiply memory cost by the number of users — infeasible at any real scalePersonalization kept as a thin, bounded overlay merged at read time (R)
Exact-prefix-only matchingA single typo produces a dead-end path with no useful completions, for a real and common class of inputA bounded secondary fuzzy/phonetic index over only the most popular queries (S)

Y. Design Trade-Offs

DecisionOption AOption BTrade-off
Ranking computationLive, per-request scan/sortPrecomputed top-K per trie node (chosen)Live computation is simpler and always perfectly fresh, but scales worst exactly where traffic is heaviest; precomputing trades some staleness for making the hottest requests the cheapest ones
Freshness mechanismReal-time insert-per-queryPeriodic batch rebuild + real-time trending overlay (chosen)Real-time insert keeps the structure always current but adds a cascading write to the hottest read path; the batch-plus-overlay split keeps the read path pure while still surfacing fast-moving trends within minutes
Trie placementSingle machine, unshardedSharded by prefix range (chosen)Unsharded is simpler to operate but caps out once the corpus exceeds one machine's memory; sharding costs a routing layer in exchange for effectively unbounded horizontal scale
PersonalizationBaked into a per-user structureThin overlay merged at read time (chosen)A per-user structure gives the cleanest personalization but multiplies memory cost by user count; a merged overlay keeps one shared, cacheable base structure at the cost of a second, smaller lookup per request
Typo toleranceNone (exact-prefix only, chosen default)Full fuzzy/edit-distance index over the entire corpusExact-prefix is fast and simple but misses a real class of common typing errors; full fuzzy search over the whole corpus would meaningfully raise both cost and lookup latency for a benefit concentrated in a minority of requests

Z. Common Bad Designs

AA. Interviewer Follow-Ups

Follow-UpDesign ChangeComponents ChangedComponents Unchanged
"How would you support this across multiple languages/locales?"A separate trie (or a separate top-level branch) per language/locale, with routing decided by the client's declared locale before the prefix-range shard lookup ever happensSharding/routing layer, offline build pipeline (now per-locale)The trie structure itself, precomputed top-K, trending overlay mechanism
"What about location-aware autocomplete, like a maps search box?"Blend the text-prefix trie with a geospatial signal — either a geohash-prefixed branch of the structure or a separate geospatial index merged in at read time, reusing Ride-Sharing's geospatial primer reasoning (Prerequisites N) rather than inventing a new spatial mechanismMerge/ranking step, a new geospatial signal sourceCore trie structure, batch rebuild pipeline, edge caching
"How do you keep offensive or private (PII-containing) queries from surfacing as suggestions?"A filtering/blocklist step added inside the offline aggregation job (O), before any query ever contributes to a node's top-K cache — kept upstream of the trie build, not applied as a per-request checkThe batch aggregation job onlyEverything downstream — the trie, sharding, caching, personalization all stay identical
"What if this needs to autocomplete structured entities — products, not just free-text queries?"AggregatedQueryCount (H) gains associated entity metadata (inventory, price, category), and ranking blends textual popularity with entity-specific signals — a genuine extension to the ranking function, not a structural rewriteCore entities, the ranking/scoring function used during the batch buildThe trie structure, sharding, and caching layers themselves

AB. Interviewer Evaluation Rubric

AC. Interview Simulation

Interviewer: Design an autocomplete/typeahead system for a search box — as the user types, suggest likely completions.
Candidate: The first thing I want to avoid is computing the ranked list live, on every keystroke — a query against the raw query log with a live sort is fine for a long, rare prefix, but for something short and common like "a" or "the," you'd be scanning and ranking millions of rows under a tight latency budget, on every one of thousands of concurrent requests per second. So the core idea is a trie, with the top-K completions precomputed and cached directly at each node, built ahead of time from aggregated query logs rather than computed per request.
Interviewer: How do you keep that precomputed data from going stale?
Candidate: I'd split that into two separate mechanisms with two different freshness targets. The bulk of the ranking comes from a periodic offline batch job — query logs get aggregated with some decay weighting so recent activity counts more, and the trie gets rebuilt and pushed out every hour or so. That's deliberately not real-time. On top of that, I'd run a lightweight streaming aggregator that tracks a short recent window — the last few minutes — and produces a small "trending" overlay per prefix. At read time, I merge the stale-but-broad base result with whatever's in that trending overlay, so a genuinely fast-moving query can surface within minutes without ever making the core structure itself real-time.
Interviewer: What happens once that trie is too big to fit on one machine?
Candidate: I'd shard it by prefix range — one server owns everything under "a" through "f," another "g" through "m," and so on. The nice thing here is that the shard key falls directly out of the data structure itself; you don't need a consistent-hashing ring, because the first character of the prefix already tells you exactly which shard to route to. The one wrinkle is that query volume isn't evenly spread across the alphabet, so I'd expect to split further on whichever ranges turn out to be hottest in practice, based on actual traffic, not assumed evenly upfront.
Interviewer: How would you personalize results without redoing all of that per user?
Candidate: I'd keep personalization completely separate from the shared trie. The base lookup and the trending overlay both stay global and cacheable exactly as they are — personalization is a small, bounded check against the user's own recent search history, done in parallel, and merged in at the very end. Baking it directly into the trie — a separate structure per user — would multiply the memory cost by the number of users, which doesn't work at scale. Keeping it as a thin overlay lets one user's own signal matter without touching the shared structure everyone else depends on.

AD. Online Solution Comparison

SourceApproachStrengthWeaknessWorth Learning
Elasticsearch Completion Suggester (official docs)Builds a Finite State Transducer (FST) — a compressed, trie-like structure — at index time from configured suggestion inputs and weights, held entirely in memory for extremely fast prefix lookups; natively supports fuzzy matching via a configurable Damerau-Levenshtein edit-distance parameterA production-grade, real confirmation that a precomputed, in-memory, trie-shaped structure built ahead of time (not queried live) is the standard real-world answer to this exact problem — and that fuzzy tolerance (S) is commonly bolted on as a configurable parameter rather than solved from scratchRebuilding the FST means reindexing the underlying suggester data — genuinely real-time updates aren't native, which is exactly why this design layers a separate trending overlay (O) rather than expecting the base structure to be liveConfirms that "precompute a prefix-optimized structure offline, keep it in memory, tolerate rebuild lag" is the real production pattern, not an interview simplification
Google Search "About autocomplete predictions" (official help documentation)Documents that predictions are generated from real searches performed by users, are explicitly influenced by trending topics and rapidly updated for breaking events, and are personalized using signals including a signed-in user's own search history and general locationA direct, official confirmation that a real, massive-scale autocomplete system separates "broad historical popularity" from "trending right now" and layers personalization on top of both — the exact three-way split this design's O and R sections are built aroundThe documentation intentionally doesn't disclose ranking internals or infrastructure — useful for confirming the shape of the problem, not for replicating an implementationConfirms trending-as-a-distinct-signal and personalization-as-an-overlay are real, named, production concerns at the largest possible scale, not interview embellishments
"Auto Complete with Redis" (Salvatore Sanfilippo / antirez, Redis creator's blog)Describes simulating a trie using a single Redis sorted set, storing every prefix of every term as a member and using range queries (ZRANGEBYLEX-style lexicographic range scans) to retrieve completions for a given prefix, avoiding the need to build and maintain an actual pointer-based tree structure in application memoryA real, widely-referenced open-source pattern for implementing trie-like prefix search using an off-the-shelf data store rather than custom infrastructure — a legitimate, lighter-weight alternative to a bespoke trie server for smaller-scale versions of this problemStoring every prefix of every term multiplies storage several-fold over a true trie's shared-prefix structure, and this approach doesn't natively give you a precomputed, ranked top-K per node (N) — ranking still has to be layered on separatelyA concrete, credible demonstration that the core "prefix range lookup" idea behind a trie can be implemented multiple different ways depending on scale — worth knowing this is a spectrum, not one single canonical implementation

AE. Interview Cheat Sheet

ProblemSearch Autocomplete / Typeahead
DifficultyIntermediate → Advanced (simple read path, genuinely hard freshness/sharding/caching surface underneath it)
Core requirementsRanked prefix completion, sub-100ms per keystroke, updates over time as trends shift, tolerant of scale-driven staleness
Central decisionTrie with precomputed top-K per node, rebuilt offline from aggregated logs, plus a real-time trending overlay, sharded by prefix range, cached aggressively at the edge for hot prefixes
Why not live rankingA live scan/sort or a live subtree traversal is slowest exactly on the short, common prefixes that account for most real traffic — precomputing inverts that relationship
Freshness splitTwo distinct mechanisms: a slow, broad, decay-weighted batch rebuild for the bulk ranking; a fast, narrow streaming overlay for genuinely trending queries
ShardingBy prefix range — the tree's own structural boundary supplies the shard key directly, no consistent-hashing ring required
CachingLayered edge/CDN → shared cache → origin shard, but only for the shared, non-personalized base ranking
Personalization & typosBoth are thin, bounded overlays merged at read time — never baked into the shared precomputed structure itself
ConsistencyEventually consistent everywhere, with no strong-consistency island at all — staleness here costs ranking quality, never correctness
Common mistakesLive prefix scans/sorts; a trie with no precomputed ranking; real-time insert-per-query into the serving structure; one unsharded trie; personalization baked into the shared cache; expecting perfect typo tolerance for free
What NOT to over-engineerA fully general fuzzy-search engine and a from-scratch ML ranking model — name both as real, bounded extensions (S, AA) rather than designing them from first principles in the room