1. URL Shortener (Bitly) Beginner → Advanced+Scale
A. Realistic Interview Prompt
Note what is not specified: custom aliases, expiration, analytics, auth, exact scale numbers, latency target. A strong candidate treats this as an invitation to negotiate scope (section E) before drawing a single box — exactly the discipline Hello Interview's format is built around: agree on requirements, build the simplest thing that satisfies them, then let each requirement earn its own piece of complexity.
B. Difficulty
Beginner-friendly entry point that escalates fast. The basic version (single server, single DB, synchronous hash) is genuinely simple — this is why it's the most common "warm-up" system design question. But it hides a surprising amount of depth once you push on any single non-functional requirement: uniqueness at scale, sub-100ms redirects, 99.99% availability, and a 1000:1 read-heavy access pattern. Interviewers use this problem specifically because the basic version is trivial — it tests whether a candidate can recognize which requirement breaks the naive design first and fix only that, rather than either freezing on a trivial problem or over-building for scale nobody asked for.
C. What This Problem Tests
- Whether you can separate "does it work" (functional correctness) from "does it survive load" (non-functional requirements) and sequence your design around the latter
- Back-of-envelope capacity estimation and turning it into real decisions (cache size, code length, sharding-or-not)
- Recognizing a read-heavy workload and architecting asymmetric scaling instead of scaling everything uniformly
- Knowing when NOT to add complexity — sharding, edge compute, and Kafka are all wrong answers if introduced before the requirement that justifies them
- CAP/PACELC reasoning: picking availability over consistency for redirects, while still guaranteeing strong uniqueness where it actually matters
D. Requirements
Functional
- A user can submit a long URL and receive a shortened URL
- A user can optionally specify a custom alias for the short code
- A user can optionally specify an expiration date/time for the short URL
- Visiting the short URL redirects the client to the original long URL
Non-Functional
- Uniqueness — every short code maps to exactly one long URL, always, even under concurrent writes
- Low latency — redirects should feel instant; target <100ms end-to-end for the redirect hop specifically (creation can tolerate more latency — it's a rare, deliberate action, not a click-path)
- High availability — 99.99% uptime (~52 minutes of downtime/year). A broken short link is a broken link everywhere it was ever shared (tweets, print media, SMS) — favor availability over strict consistency
- Scale — support on the order of 1 billion stored URLs and 100 million daily active users, with a heavily read-skewed access pattern (redirects vastly outnumber new-URL creations)
Explicitly out of scope (the "below the line" cut, unless the interviewer pulls it back in — both come back as deep dives S and R): user authentication/account management, analytics dashboards on link clicks. Calling this out loud, and defending it, is itself a signal of scoping discipline — see AA for what happens when candidates build these unprompted.
E. Clarifying Questions
| # | Question | Why It Matters | Design Decision Affected |
|---|---|---|---|
| 1 | Roughly what scale — total URLs stored, daily active users, read:write ratio? | Every later decision (code length, cache size, sharding-or-not) is a function of these numbers, not intuition | Capacity estimation (G) |
| 2 | Is a strict latency SLA given, or should I assume one? | Determines how aggressively caching/CDN layers are justified | Deep Dive 2 (N) |
| 3 | Do short codes need to be unguessable, or just unique? | Sequential counter IDs are fine for uniqueness but leak information if guessability matters | ID generation, Security (M, S) |
| 4 | Are custom aliases in scope? | Adds a second, human-driven uniqueness path alongside the generated one | Deep Dive 5 (Q) |
| 5 | Do links expire, and if so what happens to expired data — kept or purged? | Changes whether a cleanup job is needed and how cache TTLs are set | Deep Dive 5 (Q) |
| 6 | Is click analytics in scope now or a likely follow-up? | If in scope, it must not be bolted onto the synchronous redirect path | Deep Dive 6 (R) |
| 7 | Single region or multi-region from day one? | Changes whether the ID generator needs a cross-region coordination story immediately | Deep Dive 1, 3, 4 (M, O, P) |
| 8 | Is deletion/update of an existing short URL supported? | Affects the redirect status code choice (301 vs 302) and cache invalidation strategy | API design, Deep Dive 2 (I, N) |
| 9 | Do we need to defend against malicious/phishing long URLs? | A shortener that hides destinations is an attractive phishing vector | Deep Dive 7 (S) |
| 10 | Is there a budget/preference constraint (e.g. "no external dependencies," "must run on our own infra")? | Affects whether Redis/CDN/managed services are even on the table | Whole design |
F. Assumptions
- Confirmed (from prompt): shorten + redirect are the core functional loop; "at scale" was explicitly said, so capacity estimation is not optional here.
- Assumption: ~1 billion total stored URLs, ~100 million DAU, and a ~1000:1 read:write ratio — a standard, defensible scale for this problem, used consistently through this document so every later number is traceable (see G).
- Assumption: custom alias and expiration ARE in scope (they were named in the base prompt phrasing used by most interviewers for this problem) — analytics and auth are NOT, until a follow-up asks (sections R and S cover them anyway, clearly marked as extensions).
- Assumption: short codes should be reasonably resistant to enumeration (soft security requirement) even though nobody explicitly demanded encryption-grade unguessability — addressed cheaply in Deep Dive 1/7 rather than ignored.
- Assumption: single primary region in the base design (V1–V5); multi-region is introduced explicitly as an extension (Deep Dive 3 and 4), mirroring how real interviews escalate this problem.
G. Capacity Estimation (Back-of-Envelope)
Every number below is derived from the same three inputs so nothing is a "random" constant: 1B total URLs, 100M DAU, 1000:1 read:write ratio.
| Metric | Calculation | Result |
|---|---|---|
| Redirects/day | 100M DAU × ~5 redirects/user/day (assumed click rate) | ~500M redirects/day |
| Average redirect QPS | 500M ÷ 86,400s | ~5,800 QPS |
| Diurnal peak QPS | ~10× average (normal daily peak hour, not a black-swan event) | ~58,000 QPS |
| Viral-burst worst case | ~100× average (a single link goes viral — a real, distinct scenario from the daily peak) | ~580,000–600,000 QPS |
| New URLs/day | Redirects/day ÷ 1000 (the read:write ratio) | ~500,000 writes/day |
| Average write QPS | 500,000 ÷ 86,400s | ~6 QPS (trivial — writes are never the bottleneck here) |
| Storage per row | short_code(8B) + long_url(~200B avg) + user_id(8B) + created_at/expires_at(16B) + flags/padding(~270B) | ~500 bytes/row |
| Total storage @ 1B rows | 1,000,000,000 × 500 bytes | ~500 GB |
| Years to reach 1B at 500K/day | 1B ÷ (500K × 365) | ~5.5 years |
| Short-code space @ 6 chars, base62 | 626 | ~56.8 billion combinations (57× headroom over 1B) |
| Short-code space @ 7 chars, base62 | 627 | ~3.52 trillion combinations (3,520× headroom) |
The two numbers that actually drive the architecture: (1) 500GB total storage is trivially small — it fits on a single modern DB node's SSD with room to spare, which means we do not need to shard for storage capacity reasons. (2) The gap between 5,800 QPS average and 600,000 QPS viral-burst is 100×, and it's almost entirely reads — which means the entire scaling story of this system is about absorbing read spikes cheaply (caching), not about storage or write throughput. Every subsequent deep dive is a consequence of one of these two facts.
Code length: 6 vs. 7 characters — a genuine judgment call, not a rule. 6 characters already give 57× headroom over the 1B target, which is generous. 7 characters give 3,520× headroom for negligible extra cost (1 more byte × 1B rows ≈ 1GB, irrelevant at a 500GB scale) and remove any need to ever migrate code length later as the service grows past 1B. This document uses 7 characters for that long-term-headroom reason, but 6 is an equally defensible answer if you say why.
H. Core Entities
| Entity | Purpose | Key Fields | Notes |
|---|---|---|---|
ShortUrl | The core mapping — the entire system exists to create and resolve these | short_code (PK), long_url, user_id (nullable), created_at, expires_at (nullable), is_custom_alias | The ONLY entity that must be strongly consistent (see V) |
User | Optional creator/owner of a link | user_id, email | Out of scope per D unless auth is pulled back in — included here only so ownership fields have somewhere to point |
ClickEvent | One redirect occurrence, for analytics only | short_code, timestamp, referrer, geo, user_agent | Lives in a separate analytics store, never the hot-path DB — see Deep Dive 6 (R) |
I. API Design
POST /api/v1/urls
body: { long_url, custom_alias?, expires_at?, user_id? }
201 → { short_url, short_code, expires_at }
400 → invalid long_url
409 → custom_alias already taken
429 → rate limited (see S)
GET /{short_code}
302 → Location: <long_url>
404 → unknown short_code
410 → short_code expired
Why 302 (Found / temporary), not 301 (Moved Permanently) — the single most consequential API decision in this whole design:
| 301 Permanent | 302 Temporary | |
|---|---|---|
| Browser caching | Browser caches the redirect itself — after the first click, the browser never contacts our server again for that link | Browser does not persist the redirect — every click hits our server |
| Server load | Lower — repeat clicks from the same browser are free | Higher — every click is a real request, which is exactly why Deep Dive 2 (caching) exists |
| Expiration support | Broken — a browser that cached the 301 will keep "redirecting" locally even after we expire/delete the link server-side | Works correctly — expiration is enforced on every request, not just the first |
| Click analytics | Broken — we're only told about the very first click per browser, ever | Works — every click is observable server-side |
Given that expiration and click analytics are both stated or near-certain requirements, 302 is the correct choice — the extra server load it implies is the reason Deep Dive 2 exists, not a flaw to avoid by picking 301.
J. Basic / Naive High-Level Design (V1)
The simplest design that satisfies the functional requirements alone, deliberately ignoring every non-functional one for now — this is the baseline every deep dive below is measured against.
(validate, hash+base62, write)"] S -- "2 . INSERT short_code, long_url" --> DB[("Relational DB
single node")] C -- "3 . GET /{short_code}" --> S S -- "4 . SELECT long_url" --> DB S -. "5 . 302 redirect" .-> C
V1 — everything on one server, one database. Satisfies function, fails every NFR (see callout below).
V1 mechanics: on create, hash the long_url (e.g. MD5), take the first 7 characters of the base62-encoded hash as the short_code, check the DB for a collision, and on collision append a salt and rehash. On redirect, a single indexed SELECT by short_code, then a 302.
Why this is fine as a starting point and exactly where it breaks: it satisfies all four functional requirements. It fails uniqueness guarantees cleanly (hash collisions require detect-and-retry, not prevention — M), fails latency the moment traffic exceeds what one DB node's disk I/O can serve (N), fails availability the moment that one server or DB dies (P), and fails scale the moment write and read load diverge enough that one fleet can't serve both efficiently (O). Each of those four failure points maps to exactly one deep dive below — this is not incidental, it's the whole point of the methodology.
K. Data Model & Database Choice
| Field | Type | Notes |
|---|---|---|
short_code | VARCHAR(7) | Primary key — gives us indexing and the uniqueness constraint for free, see M and Q |
long_url | VARCHAR(2048) or TEXT | URLs can legally be long; don't under-allocate |
user_id | BIGINT, nullable | Nullable because anonymous shortening is allowed |
created_at | TIMESTAMP | — |
expires_at | TIMESTAMP, nullable | Null = never expires |
is_custom_alias | BOOLEAN | Distinguishes the two write paths in Q |
Why a relational DB (Postgres/MySQL) over a pure key-value/NoSQL store: the access pattern is a single-key point lookup by short_code — something almost any datastore does well, so the "which DB" decision is genuinely low-stakes here (this matches what every source surveyed in AD independently concludes). A relational DB is chosen for the free UNIQUE constraint (does real work for us in M and Q — no application-level race condition to reason about) and because 500GB/1B rows (from G) is nowhere near the scale where a relational DB's vertical limits bite. A key-value store (DynamoDB, Cassandra) is an equally valid choice if the team already operates one at scale and wants to skip capacity planning for a relational primary — call this out as a legitimate alternative, not a mistake.
L. End-to-End Flow (Basic)
M. Deep Dive 1 — Ensuring Short-Code Uniqueness at Scale
Requirement this addresses: Uniqueness (D), under concurrent writes, without every write paying for a DB round-trip just to detect a collision.
| Approach | How It Works | Why Good | Why Bad |
|---|---|---|---|
| Hash (MD5/SHA-256) + truncate + collision retry | Hash the long_url, base62-encode, take first 7 chars; on DB collision, salt and rehash | Stateless — purely a function of input, no coordination needed; identical long URLs naturally dedupe | Collision probability rises with N (birthday paradox); every write pays a DB check-then-maybe-retry loop even in the common case of no collision — the exact "detect and fix" pattern the counter approach below eliminates by construction; conflates dedup with per-user ownership unless salted |
| Unique counter + base62 encode (chosen) | A single monotonically increasing integer ID is issued per URL; base62-encoding an integer is bijective, so distinct integers → distinct codes, always | Zero collisions by construction — no DB check, no retry loop, O(1); deterministic and simple to reason about | Needs one coordinated source of truth for "next ID" — a distributed-systems problem in itself; codes are sequential → guessable/enumerable (addressed in S) |
| Snowflake-style ID (timestamp + worker-id + sequence) | Embed uniqueness into the ID's bit structure itself instead of a central counter | Removes the central counter service as a dependency entirely; well-proven at Twitter-scale | Longer IDs (64 bits ≈ 11 base62 chars, vs. 7 for a pure counter); requires synchronized clocks (NTP) across ID-generator nodes |
Why the counter, not the hash, is the right default here: the hash approach's core weakness isn't collision math (at 1B/3.5T fill, collisions are genuinely rare) — it's that it makes the DB unique-check-on-write mandatory for correctness, coupling every write to a synchronous round-trip that a counter-based design simply doesn't need. The counter's weakness (needing a coordinated ID source) is solved cheaply with Redis, below.
Implementation — Redis INCR as the counter: Redis processes commands single-threaded, so INCR is atomically race-free with zero application-level locking. Redis comfortably handles 100K+ ops/sec on one node — far more than our ~6 QPS average write load (G) ever needs, even accounting for the ~58/sec peak.
Optimization — counter batching: instead of one Redis round-trip per URL created, each Write Service instance calls INCRBY 1000 once, gets back a starting value, and hands out that entire local range to incoming requests without touching Redis again until the batch is exhausted. Cuts Redis network round-trips by ~1000×. The cost: a crashed instance "wastes" its unused batch — acceptable, since the ID space (3.52 trillion at 7 chars) is enormous and we're nowhere near exhausting it.
(no network call per URL)" --> E["short_code"] E -- "4 . INSERT short_code, long_url" --> DB[("Primary DB")]
Counter batching: one Redis round-trip serves 1,000 URL creations.
Multi-region extension: give each region a disjoint ID range (e.g. us-east owns block 0–999,999,999; eu-west owns 1,000,000,000–1,999,999,999) so regions never coordinate with each other for uniqueness. This trades a small amount of ID-space efficiency for full write availability during a cross-region network partition — an explicit, deliberate Availability-over-Consistency choice consistent with the Uptime NFR (see V).
N. Deep Dive 2 — Ensuring Fast, Cheap Redirects
Requirement this addresses: Low latency (<100ms) under a ~1000:1 read-heavy load that peaks at ~600K QPS (G). Each layer below solves exactly what the previous layer can't — don't skip straight to the last one.
Layer 1 — DB indexing
short_code as the primary key gives an automatic B-tree index → O(log n) lookup instead of a full scan. Necessary, not sufficient: even an indexed lookup means a disk seek. A single SSD-backed DB node tops out around ~100K IOPS — to serve a 600K QPS viral burst from disk alone would need 6 fully-loaded replicas just for redirects, which is fragile and wasteful for what should be a cheap operation.
Layer 2 — In-memory cache (Redis/Memcached), cache-aside
Check cache first (~0.1ms, ~1M ops/sec/node) → hit avoids the DB entirely; miss falls through to the DB and populates the cache for next time. Link-click traffic is famously power-law/Zipfian — a small fraction of links (a viral tweet, a marketing campaign) account for the overwhelming majority of clicks — so a cache sized to hold just the hot set absorbs the vast majority of the 600K QPS burst, cutting DB load by 90%+. The cache's real job here is load-shielding the DB, not just shaving milliseconds.
Why cache-aside, not write-through: write volume (~6 QPS) is negligible next to read volume — pre-populating the cache on every write would waste cache memory on links that may never be clicked, whereas populating on first read guarantees the cache only ever holds things actually being requested. TTL is capped at min(expires_at - now, 24h) so entries never outlive their own expiration and the cache doesn't grow unbounded; eviction policy is LRU so the genuinely hot working set stays resident.
Layer 3 — CDN / Edge compute (Cloudflare Workers, Lambda@Edge)
Push the redirect decision itself to edge PoPs geographically close to users. For a redirect specifically, network round-trip time to a centralized origin (50–150ms for a global user base) dominates over compute time — so physical proximity matters more than a faster origin. Why NOT reach for this first: cache invalidation now has to fan out across dozens/hundreds of edge locations (a harder consistency problem — a deleted/expired link may briefly still resolve at a stale edge node); edge runtimes have real constraints (execution time, memory, limited libraries); debugging a distributed edge function is harder than one origin; cost per request is typically higher. Use this as the last layer, once origin caching is solid and the interviewer specifically pushes on a global user base still not feeling fast enough — a Redis cache plus regional deployment already solves the bulk of the latency problem for far less complexity.
Three layers, each catching what the previous one misses: edge → cache → replica.
O. Deep Dive 3 — Scaling to 1B URLs and 100M DAU
Requirement this addresses: Scale (D) — but note storage scale and throughput scale are two different problems with two different fixes, and conflating them is the most common mistake on this problem (see Z).
Storage is not the bottleneck. 500GB (from G) fits comfortably on one modern DB node's NVMe storage with room to spare. We do not need to shard the database for storage capacity at this scale. The actual bottlenecks are read throughput and availability — so the correct scaling moves are caching (N) and service separation (below), not horizontal partitioning, until the design is one or two orders of magnitude beyond this stated scale.
Service separation — Read Service vs. Write Service: split into two independently-scaled, stateless fleets behind a load balancer, because their load profiles are wildly asymmetric (1000:1 from G) — coupling them means either over-provisioning writes or starving reads. Both are stateless (no sticky sessions), so plain round-robin load balancing works, and any instance can die with zero client-visible state loss.
DB read replicas: the primary handles writes; N read replicas absorb whatever read traffic misses the cache (a small percentage, given N's cache hit-rate argument). Async replication is fine — a few seconds of staleness is imperceptible for a URL that was created seconds ago and is only now being clicked by someone else.
The one sharp edge case worth raising unprompted — read-your-writes: if a user creates a link and immediately clicks it themselves before replication catches up, a stale replica read could 404. Mitigation: populate the cache synchronously at write time for just-created links (a deliberate, narrow exception to the cache-aside-only rule) so the very next read never needs the replica at all. Raising this proactively is a strong signal — it's exactly the kind of gap Hello Interview's own seniority rubric (AB) rewards at the senior/staff level.
When would sharding actually be justified? If scale grew another 10–100× (tens of billions of rows), or if a single write region genuinely became the write bottleneck (it won't at ~6 QPS average). If/when it is justified, shard by hash(short_code) — a redirect lookup routes directly to the one shard that owns that code, with zero cross-shard fan-out. Sharding by created_at or user_id would be wrong here specifically because it doesn't match the redirect hot path's access key.
P. Deep Dive 4 — High Availability & Fault Tolerance
Requirement this addresses: 99.99% uptime (D). Go component by component — availability is never one decision, it's the absence of any single point of failure anywhere on the critical path.
- Primary DB: primary-replica replication with automated failover (Patroni for self-managed Postgres, or a managed Multi-AZ offering). Async replication is chosen over sync — losing at most a few seconds of very-recently-created (not-yet-clicked) mappings on a rare failover is an acceptable trade for keeping normal-path write latency low, consistent with the Availability-leaning NFR.
- Redis counter: a single instance is a SPOF. Use Redis Sentinel (auto-promotes a replica on failure) or Redis Cluster at larger scale. Residual risk, stated honestly: if Redis loses very recent state before replicating (e.g., a crash right after handing out a batch but before persisting it), a batch of IDs could theoretically be reissued. This is caught, not ignored — the DB's
UNIQUEconstraint onshort_codeis the real source of truth; a reissued ID just fails one INSERT, which retries with a fresh ID. Two independent uniqueness guarantees, not blind trust in Redis alone. - Cache layer: losing the cache (restart, cold start) is a performance problem, never a correctness one — traffic simply falls through to DB replicas temporarily. Watch for a cache-stampede if many nodes cold-start at once; mitigate with staggered restarts or request coalescing on hot keys. The dependency direction here is healthy: the cache can vanish without losing data, because the DB — not the cache — is the source of truth.
- Multi-region: active-active for reads (any region serves redirects from its local cache/replica); writes are regional-scoped via the disjoint counter ranges from M. A full region outage loses that region's write capability temporarily, while reads for already-created links keep working globally via cross-region async replication.
Q. Deep Dive 5 — Custom Aliases & Expiration
Custom alias uniqueness: a naive "SELECT to check if taken, then INSERT if free" has a race window between the check and the insert under concurrency — two requests for the same alias can both pass the check. The fix is the same lesson as concurrent spot allocation in LLD: don't check-then-act, let the datastore's atomicity do the work. Attempt the INSERT directly; the UNIQUE constraint on short_code rejects the second writer, and the app returns 409 Conflict. This is the one place a DB round-trip on write is genuinely unavoidable — a human picked the string, not us, so there's no counter to fall back on.
Expiration enforcement: the cache TTL is already set to min(expires_at - now, max_ttl) (N), so most expired entries simply age out of the cache on their own. As defense in depth against clock skew or TTL edge cases, the redirect handler explicitly re-checks expires_at on any DB-sourced read before returning 302, returning 410 Gone if it's passed.
Cleanup of expired rows: don't delete synchronously on the read path — that turns a cheap redirect into a write, the exact anti-pattern Deep Dive 6 warns against for analytics. Either run a periodic (e.g. hourly) background sweep that batch-deletes expired rows, or simply leave them — at 500GB/1B rows (G), storage is cheap enough that "leave and archive opportunistically" is a perfectly defensible answer, since the expires_at check already makes them unreachable regardless of whether the row physically still exists.
R. Deep Dive 6 — Analytics / Click Tracking
Why this matters even though D marked it out of scope: it is almost always the first follow-up an interviewer adds, specifically because it's the most common trap on this problem.
The trap: naively incrementing a counter or inserting a row into the same primary DB synchronously on every redirect. This silently turns every read request into a write request against the primary datastore — at 600K QPS peak redirect load (G), that's 600K writes/sec you never designed for, and it defeats the entire point of the cache-heavy read path built in N. This single mistake — conflating "read the mapping" with "record that a read happened" — is the most common failure mode candidates hit on this exact follow-up (see Z).
The fix — fire-and-forget async pipeline: on every redirect, emit a lightweight event to a message queue/log (Kafka/Kinesis) and return the 302 immediately, without waiting for the event to be durably logged. A stream consumer aggregates click counts (by short_code, by day, by geo/referrer) and writes the aggregates into a separate analytics datastore (columnar/OLAP-friendly — ClickHouse, Druid, or even a separate Postgres table set) that the redirect hot path never touches or waits on.
OLAP")] Dash["GET /urls/:code/stats"] --> AD
Analytics never sits on the redirect's critical path — the 302 doesn't wait on the queue.
An occasional lost click count on a queue hiccup is acceptable — analytics is an "eventually roughly right" requirement, unlike the URL mapping itself, which must be exactly right (see V). A separate GET /urls/{code}/stats endpoint reads only from the analytics store, never the hot-path cache/DB.
S. Deep Dive 7 — Security & Abuse Prevention
Enumeration / guessability: the counter approach chosen in M produces sequential codes — if code N exists, an attacker knows code N+1 probably exists too, enabling scraping of "unlisted but not secret" links. Fix without giving up the collision-free counter: reversibly obfuscate the counter before base62-encoding it (XOR/permute with a fixed secret, or a small keyed Feistel-style bit permutation) so codes look random but remain a strict 1:1 function of the counter — no collision risk introduced, no DB check needed, and decode is O(1) with the secret. This is a deliberately different trade-off than just switching to a hash (M) — a hash reintroduces the collision-detection cost we specifically avoided; the obfuscated counter gets unguessability and collision-freedom at once.
Malicious/phishing long URLs: a shortener that hides its destination is an attractive phishing vector. Mitigate by checking submitted long_urls against a threat-intel blocklist (e.g. Google Safe Browsing API) at creation time, and re-scan periodically afterward — a domain can turn malicious after the link was already created, so a one-time check at creation isn't sufficient. Re-scanning runs on the same async, non-blocking shape as the analytics pipeline in R.
Abuse rate limiting: a token-bucket or sliding-window limiter (per-IP and/or per-API-key) sits at the edge/API gateway in front of the Write Service, preventing one bad actor from mass-creating links or hammering the counter/DB — a distinct concern from the general-capacity scaling in O, which assumes well-behaved aggregate traffic.
API Gateway"} RL -- "under limit" --> WS["Write Service"] RL -- "over limit" --> C2["429 Too Many Requests"] WS -- "long_url" --> SB{"Safe-Browsing /
Threat-Intel Check"} SB -- "flagged" --> Reject["Reject / flag for review"] SB -- "clean" --> DB[("Primary DB")]
T. Full Solution at Scale — Final Architecture
Every deep dive above composed into one system. Nothing here is new — this section is deliberately just the sum of M through S, which is the point: the design evolved one justified piece at a time (see U), never all at once.
(N — last-mile latency)"] Edge -- "cache miss" --> LB["Global Load Balancer"] LB --> RL{"Rate Limiter / API Gateway (S)"} subgraph WRITE [" Write Path (O) "] direction TB WS["Write Service
(stateless, autoscaled)"] Redis[("Redis Counter
batched INCR — Sentinel/Cluster (M, P)")] SB{"Safe-Browsing /
Threat-Intel Check (S)"} WS -- "1 . INCRBY 1000" --> Redis WS -- "2 . check long_url" --> SB end subgraph READ [" Read Path (N, O) "] direction TB RS["Read Service
(stateless, autoscaled)"] Cache[("Redis / Memcached
cache-aside")] RS -- "1 . lookup" --> Cache end subgraph STORAGE [" Storage (K, P) "] direction TB Primary[("Primary DB
short_code | long_url")] Replica[("DB Read Replicas")] Primary -. "async replication" .-> Replica end subgraph ASYNC [" Async Pipeline (R, S) "] direction TB Kafka[("Kafka / Kinesis")] Agg["Stream Aggregator"] AnalyticsDB[("Analytics DB — OLAP")] Kafka --> Agg --> AnalyticsDB end RL --> WS RL --> RS WS -- "INSERT (clean only)" --> Primary WS -. "click/creation events" .-> Kafka Cache -- "miss" --> Replica RS -. "click events" .-> Kafka RS -. "302" .-> U
Full solution at scale — the composition of deep dives M through S, nothing new introduced here.
U. Design Evolution
- V1 — Naive monolith (J): one server, one DB, synchronous hash + collision retry.
- V2 — Uniqueness fixed (M): replace hash+retry with a Redis-backed counter + base62 encoding; batching added to cut Redis round-trips.
- V3 — Latency fixed (N): cache-aside Redis layer in front of the DB for redirects.
- V4 — Throughput/availability fixed (O, P): split into Read/Write services; add DB read replicas and primary failover.
- V5 — Global scale (N, M, P): CDN/edge redirect layer; multi-region disjoint counter ranges; active-active reads.
- V6 — Analytics added (R): async event pipeline (Kafka → aggregator → analytics DB), deliberately outside the hot path.
- V7 — Security hardened (S): obfuscated (non-sequential-looking) codes, malicious-URL scanning, rate limiting.
What stays stable across every version: the public API contract from I — POST /urls and GET /{short_code}. That stability is the payoff of treating ID generation, caching, and storage as swappable internals from the start — clients never notice the system evolving underneath them, the same principle the LLD handbook's Parking Lot facade demonstrates at the class level.
V. Consistency Model & CAP Trade-offs
The system is deliberately AP-leaning (Availability over Consistency) almost everywhere: a redirect should keep working even if a replica is slightly stale or a node is down — an unavailable redirect is a publicly broken link, arguably worse for reputation than a rare stale read. Writes tolerate a small async-replication lag; the cache tolerates being wrong for at most one TTL window.
The one deliberate exception — strong consistency for uniqueness itself. short_code uniqueness is enforced via the primary DB's UNIQUE constraint (strongly consistent on a single primary) plus the Redis counter's atomic INCR — the one invariant the whole system cannot be loose about. This is a hybrid, not a contradiction: strong consistency for the one narrow guarantee that actually matters (two links must never silently collide), eventual consistency for everything else (replica reads, cache freshness, analytics counts).
W. Failure Scenarios
- Redis counter node down: Sentinel/Cluster promotes a replacement (P); worst case, a small batch of already-issued-but-uncommitted IDs gets reissued — caught by the DB's UNIQUE constraint, not silent corruption (M).
- Cache cluster down: traffic falls through to DB replicas; a performance problem, never a correctness one (P) — watch for stampede on simultaneous cold restart.
- Primary DB down: automated failover promotes a replica; a few seconds of very-recent writes may be lost (accepted trade-off, V).
- Region outage: that region's writes pause (or reroute to another region's counter block); reads for existing links keep working globally via cross-region replication (P).
- Read-your-write race: creator clicks their own link before replication catches up — mitigated by synchronously warming the cache on write for new links (O).
- Custom alias race: two users request the same alias simultaneously — resolved by the DB's UNIQUE constraint rejecting the loser, not an app-level check (Q).
- Long URL flagged malicious after creation: periodic re-scan catches domains that turned malicious post-creation, not just at write time (S).
X. Bottlenecks & Scaling Boundary
| Stage | Actual Bottleneck | Fix Applied |
|---|---|---|
| V1 naive | Single server/DB — any load at all | Everything below |
| Post-cache | DB disk IOPS on cache miss | Cache-aside absorbs 90%+ of reads (N) |
| Post-service-split | Cache memory size vs. working-set size | Size cache to the Zipfian hot set, not total dataset (N) |
| Post-multi-region | Cross-region coordination for ID uniqueness | Disjoint counter ranges per region (M) |
| Hyperscale (10-100× stated scale) | Single DB's total write/storage ceiling | Shard by hash(short_code) — only justified here, not sooner (O) |
At no point in the stated 1B/100M-DAU scale (G) is database sharding actually necessary — it appears only at the bottom row, as a hyperscale extension. Reaching for it earlier is the single most common over-engineering mistake on this problem (Z).
Y. Design Trade-Offs
| Decision | Option A | Option B | Trade-off |
|---|---|---|---|
| ID generation | Hash + collision retry | Counter + base62 (chosen) | Hash needs no coordination but pays a DB check per write and gets slower as fill increases; counter is O(1) and collision-free but needs one coordinated ID source |
| Redirect status | 301 Permanent | 302 Temporary (chosen) | 301 is cheaper (browser-cached) but breaks expiration and analytics; 302 costs more server load but both features require it |
| DB replication | Synchronous | Async (chosen) | Sync guarantees zero data loss on failover but adds write latency and needs quorum; async keeps writes fast, accepts a few seconds of loss risk on a rare failover |
| Cache population | Write-through | Cache-aside (chosen) | Write-through keeps cache always warm but wastes memory on never-clicked links; cache-aside only caches what's actually requested |
| Redirect edge layer | Origin-only | CDN/Edge compute | Origin-only is simpler to reason about and debug; edge cuts network RTT for global users at the cost of harder cache invalidation and higher $ cost |
| Sharding | Single primary + replicas (chosen at stated scale) | Shard by hash(short_code) | Single primary is simpler and sufficient at 500GB/1B rows; sharding only pays off 10-100× beyond stated scale |
Z. Common Bad Designs
- Hash + DB-check-on-every-write at scale. Looks natural (it's the first idea everyone has), but it makes every single write pay a synchronous collision-check round-trip forever, and collision handling logic grows uglier as fill increases. Fix: the counter-based approach in M, which is collision-free by construction.
- Writing analytics synchronously into the hot-path DB. The single most common mistake on the analytics follow-up specifically — it turns a 600K QPS read spike into an equally large write spike on the primary datastore. Fix: async event pipeline, entirely decoupled from the redirect response (R).
- Sharding the database "because 1 billion rows sounds big." 1B rows at ~500 bytes each is 500GB — trivially small for a single modern DB node. Sharding here adds cross-shard complexity for a problem that doesn't exist yet. Fix: recognize that the actual constraint at this scale is read QPS, not storage (O, X).
- Defaulting to 301 redirects "because they're faster." They are faster for repeat clicks, but silently break both expiration and click analytics, both of which are stated requirements. Fix: 302, and treat the extra server load it implies as the reason the caching deep dive exists (I, N).
- Single Redis instance for the counter with no Sentinel/Cluster. Works fine in a demo, is a silent SPOF in production — a NFR the availability target explicitly rules out. Fix: P.
- Check-then-insert for custom alias uniqueness. Has a race window between the SELECT and the INSERT under concurrency. Fix: attempt the INSERT directly and let the UNIQUE constraint reject losers (Q) — same lesson as concurrent resource allocation in LLD.
- Eagerly deleting expired rows on the read path. Turns a cheap redirect into an extra write for zero user-facing benefit, since the
expires_atcheck already makes the row unreachable. Fix: periodic background sweep, or don't delete at all (Q).
AA. Interviewer Follow-Ups
| Follow-Up | Testing | Design Change | Components Changed | Components Unchanged | Pattern/Trade-off |
|---|---|---|---|---|---|
| "A single link just went viral — 1M clicks in an hour." | Distinguishing diurnal peak from a black-swan spike | Cache absorbs it almost entirely since it's one hot key; if a single Redis node still saturates, replicate that one key across multiple cache nodes ("hot-key fan-out") | Cache tier | DB, ID generation | Validates G's distinction between diurnal peak (~58K QPS) and viral burst (~600K QPS) |
| "Support link previews (fetch page title/og:image)." | Recognizing scope creep vs. a real new component | Async background fetcher hits the long_url once at creation time, caches metadata separately — never on the redirect hot path | New async worker, new metadata table | Read Service, redirect flow | Same "don't touch the hot path" principle as analytics (R) |
| "User wants to delete their link (GDPR/right-to-be-forgotten)." | Consistency of deletion across cache + CDN edge | DELETE endpoint removes the DB row and explicitly invalidates cache/edge entries rather than waiting for TTL | Write Service, cache invalidation logic | ID generation | Active invalidation vs. passive TTL expiry — a real trade-off surfaced by this exact follow-up |
| "No external dependencies allowed — no Redis." | Whether the candidate over-relied on one tool | Replace Redis counter with a DB auto-increment sequence (accept it becomes a write bottleneck/SPOF sooner) or a Snowflake-style generator (M) | ID generation only | Everything else | Validates that M's alternatives section wasn't decorative |
| "Support bulk shortening via a CSV upload." | Recognizing a batch workload shouldn't reuse the single-URL hot path | Separate batch endpoint/queue that calls the same counter-batching logic in a loop, async, with a completion callback | New batch API, reuses ID generation | Redirect path entirely | Reuse the counter-batching mechanism (M) rather than inventing a new ID scheme for batches |
| "Two regions raced to create the same custom alias." | Whether the DB constraint story holds across regions, not just within one | Custom aliases must be validated against a globally-consistent store (not per-region), or accept last-writer-wins with the loser getting a 409 on next check | Custom alias write path | Counter-based codes (unaffected — disjoint ranges already prevent this for them) | Exposes that the multi-region disjoint-range trick from M doesn't automatically extend to human-chosen aliases |
| "Migrate everyone from 6-char to 7-char codes without downtime." | Backward-compatible schema evolution | New codes issued at 7 chars going forward; old 6-char codes keep resolving unchanged since length isn't validated on read, only on generation | ID generation only | Redirect/read path (already length-agnostic) | Good design didn't hardcode a fixed length anywhere on the read path — a subtle but real detail |
AB. Interviewer Evaluation Rubric
- Weak candidate: jumps straight to a single server/DB design with no capacity estimation; never distinguishes read from write load; treats "at scale" as a buzzword rather than a set of numbers to derive decisions from.
- Mid-level: produces a working high-level design with the basic shorten/redirect flow; recognizes uniqueness needs solving and proposes hashing or a counter; understands why 302 over 301; mentions caching if prompted.
- Senior: proactively identifies the read-heavy skew and structures the design around it before being asked; articulates the hash-vs-counter trade-off unprompted; details caching strategy including TTL/expiration interplay; justifies the database choice instead of defaulting to one; separates read/write services for the asymmetric load; understands Redis counter scaling and batching.
- Staff+: discusses multi-region counter partitioning, Redis failover mechanics, and the read-your-writes edge case unprompted; addresses enumeration/security without being asked; shows product judgment on custom-alias collisions and expiration cleanup; can clearly state where the design would need to change 10-100× beyond stated scale and why it doesn't need to change sooner.
Rubric adapted from the seniority framing used by Hello Interview's Bitly breakdown, restated in this document's own words.
AC. Interview Simulation
AD. Online Solution Comparison
| Source | Approach | Strength | Weakness | Worth Learning |
|---|---|---|---|---|
| Hello Interview — Problem Breakdown | Requirements-first, then deep dives per NFR (uniqueness, latency, scale); explicit seniority rubric | Cleanest requirement-to-deep-dive traceability of any source reviewed; explicit mid/senior/staff expectations | Lighter on multi-region and security specifics than a full production design would need | The core methodology this entire document follows — basic design first, then let each requirement earn its complexity |
| Hello Interview — Answer Key | Condensed final architecture with all components named explicitly | Good checklist of the exact components a final answer should include | Less narrative reasoning than the breakdown page — better as a reference than a teaching tool | Cross-checking your own final diagram against theirs component-by-component |
| ByteByteGo | Hash+collision vs. base62-of-unique-ID compared explicitly; distributed unique-ID generator section; bloom filters mentioned for collision checks | Most rigorous treatment of the ID-generation trade-off specifically; concrete capacity numbers | Less focus on the read/write service-separation and multi-region angle | The base62-of-a-globally-unique-ID framing, and the bloom-filter idea for fast negative collision checks if you do choose hashing |
| Educative — Grokking (TinyURL) | Detailed capacity estimation walkthrough (100M URLs/day style numbers), simple 3-column schema, load balancer + cache + DB layering | Very approachable capacity-estimation pedagogy for beginners | Doesn't go deep on multi-region, security, or the analytics-pipeline trap | The step-by-step capacity math structure, useful as a template for any HLD problem, not just this one |
| System Design Handbook | Production-framing (SLAs, phishing/abuse angle) rather than pure-interview framing | Best coverage of the security/abuse angle (malicious URLs, rate limiting) among sources reviewed | Less rigorous on the ID-generation trade-off than ByteByteGo or Hello Interview | The framing that a URL shortener is a real abuse vector, not just a caching exercise |
Additional cross-referenced sources (search-level only, not deep-fetched): GeeksforGeeks, Grokking-System-Design (GitHub), System Design School.
AE. Interview Cheat Sheet
| Problem | URL Shortener (Bitly) |
| Difficulty | Beginner → Advanced+Scale |
| Core requirements | Shorten (+custom alias, +expiry) and redirect; uniqueness, <100ms redirects, 99.99% uptime, 1B URLs/100M DAU at ~1000:1 read:write |
| Key numbers | ~5,800 avg / ~58K peak / ~600K viral-burst redirect QPS; ~6 write QPS; ~500GB total storage; 7-char base62 codes |
| Core entities | ShortUrl, User (optional), ClickEvent (separate store) |
| ID generation | Redis atomic counter + base62 encode + batching, over hash+collision-retry — collision-free by construction |
| Fast redirects | DB index → cache-aside (Redis) → CDN/edge, in that order, each layer justified by what the previous one can't handle |
| Redirect status code | 302, not 301 — 301 breaks expiration and analytics |
| Most important decision | Recognizing storage (500GB) is a non-issue and the real scaling problem is read QPS — don't shard prematurely |
| Consistency model | AP overall (availability-first); strong consistency only for short_code uniqueness itself |
| Top follow-ups | Viral spike, link previews, deletion/GDPR, no-external-deps constraint, bulk upload, multi-region alias race, code-length migration |
| Common mistakes | Hash+DB-check on every write; synchronous analytics writes; premature sharding; defaulting to 301; single-instance Redis counter; check-then-insert for aliases |
| What NOT to over-engineer | Sharding, CDN/edge, Kafka analytics — all correct eventually, all wrong if introduced before the requirement that justifies them |