7. Video Streaming (YouTube) Advanced

A. Realistic Interview Prompt

The defining shape of this problem is that upload and playback are almost entirely separate systems sharing only a video ID — upload is a slow, resource-intensive, asynchronous pipeline (transcoding into many formats can take longer than the video's own runtime); playback is a latency-critical, massively-fanned-out read path that must adapt to wildly varying device and network conditions. Conflating the two into one synchronous flow is the single most common mistake.

B. Difficulty

Advanced — there's a simple-sounding naive answer ("store the file, serve the file"), but it fails for a reason most candidates haven't personally encountered (files formatted for one device stutter or fail entirely on another), making the core insight — adaptive bitrate streaming — genuinely non-obvious rather than a scale-driven afterthought like most of this document's other V1-to-V2 transitions.

C. What This Problem Tests

• Understanding that "upload" and "watch" are different systems with different latency/throughput/consistency needs, sharing only an identifier
• Knowledge of adaptive bitrate streaming (HLS/DASH) as the actual real-world answer to "smooth playback across devices and networks," not just "serve a video file"
• Judgment about what work belongs in a synchronous upload-acknowledgment path versus an asynchronous post-processing pipeline (transcoding) — the same async-decoupling instinct tested in prior problems, applied to a much heavier per-item workload
• CDN/edge-caching reasoning at a scale where the "hot" content (viral videos) and "cold" content (the long tail) have wildly different serving economics
• Comfort treating view-counting and recommendations as separable, approximate systems layered on the core storage/delivery pipeline, not part of its critical path

D. Requirements

Functional

1. Users can upload a video; the platform processes it into a form suitable for streaming
2. Users can watch any processed video, with playback quality adapting to their device and network conditions
3. Users can search for videos by title/metadata and see basic engagement info (view count)

Non-Functional

• Massive read-heavy scale — this document uses 500 hours of video uploaded per minute and 5 billion video views per day as the standard illustrative input, the same order of magnitude widely cited for YouTube's own real-world scale
• Smooth playback across heterogeneous devices/networks — buffering/stalling is the core failure mode this system exists to avoid, more than any specific latency number
• High upload durability — losing a user's uploaded video is a far worse failure than a delay in it becoming watchable
• Global low-latency delivery — a viewer anywhere in the world should get near-local playback latency, which a single-origin server structurally cannot provide

Out of scope: live streaming (a materially different, lower-latency real-time pipeline), monetization/ads, detailed content moderation.

E. Clarifying Questions

F. Assumptions

• Confirmed: upload, processing, on-demand playback with adaptive quality, search, and view counts are the core functional loop.
• Assumption: on-demand (pre-recorded) video only — live streaming is explicitly out of scope (E's clarifying question 1), since it requires a fundamentally different low-latency pipeline that would double the scope of this document's treatment.
• Assumption: a video becomes watchable once a baseline quality level finishes transcoding, with higher-quality levels continuing to process in the background — the standard, most-cited approach to the "how fast must upload-to-watchable be" trade-off.
• Assumption: approximate, slightly-delayed view counts are acceptable (E's clarifying question 3) — this single assumption is what makes the view-counting deep dive tractable without a heavyweight strongly-consistent counter.
• Assumption: a full personalized recommendation engine is named as a real-world extension (S) but not designed as a complete ML system — mirrors how ranking was treated in the News Feed problem.

G. Capacity Estimation

H. Core Entities

I. API Design

POST /v1/videos/upload-url        -> { upload_url, video_id }   (pre-signed direct-to-blob-store URL, M)
POST /v1/videos/{id}/complete     -> triggers transcoding pipeline (N)
GET  /v1/videos/{id}/manifest     -> { variants: [...], segment_urls: [...] }  (O)
GET  /v1/videos/{id}               -> { title, view_count (approx), status }
GET  /v1/search?q=                -> [{ video_id, title, ... }]

Why a pre-signed upload URL directly to blob storage, not through the application server: a video file can be gigabytes; routing that entire payload through an application server adds a slow, expensive hop for zero benefit — a pre-signed URL lets the client upload directly to the blob store, with the application server only involved before (issuing the URL) and after (receiving a "complete" notification) the actual byte transfer. The same "don't put large binary payloads through your application tier" principle used for media in News Feed and Chat System, just applied to the upload direction instead of only the read/serve direction.

J. Basic / Naive High-Level Design (V1)

The simplest thing that satisfies the functional requirement in isolation: store the uploaded file as-is, and serve that exact same file directly to every viewer on request.

V1 — one file, one format, served identically to every viewer regardless of device or connection.

Why this breaks immediately, not eventually: a single fixed-resolution, fixed-bitrate file cannot serve both a phone on a weak cellular connection and a desktop on fiber well — the phone will stall/buffer trying to download a high-bitrate file faster than its connection allows, or the desktop viewer gets an unnecessarily low-quality experience if the file was encoded low to accommodate weak connections. Unlike most of this document's other V1s (which work fine at small scale and degrade only under load), this V1 delivers a visibly broken experience for a large fraction of real-world viewers on day one, regardless of traffic volume — the core problem here is heterogeneity, not scale.

K. Data Model & Store Choice

Raw uploads and transcoded variants: blob storage (S3-class object storage) — video files are large, write-once (per variant), and read via range requests during playback, exactly the access pattern object storage plus a CDN in front of it (P) is built for.

Video/variant/manifest metadata: a relational or wide-column store keyed by video_id, holding titles, status, and pointers to each variant's blob location — small, structured, frequently queried data that belongs in a proper database, kept entirely separate from the large binary payloads it points to, the same metadata/blob split used throughout this document (News Feed's media, Chat System's attachments, Web Crawler's page content).

L. End-to-End Flow — Upload to Watchable

M. Deep Dive 1 — The Upload Pipeline

Requirement this addresses: the upload-durability NFR (D) — this is the write path, and it must never lose a user's uploaded content even though the file is large and the subsequent processing is slow.

Why direct-to-blob-store upload via a pre-signed URL, not through the API server: already introduced in I — worth restating here as the actual mechanism, not just the API shape. The API server's only job is issuing a short-lived, scoped credential (the pre-signed URL) and later receiving a lightweight "upload finished" notification; the multi-gigabyte transfer itself never touches application-tier compute at all, meaning the upload path's scalability is entirely the blob store's problem, not the API fleet's.

Chunked/resumable upload: for large files over unreliable networks, the upload itself is typically chunked (multipart upload, a standard blob-store feature) so a dropped connection mid-upload only requires resuming the missing chunks, not restarting a multi-gigabyte transfer from zero — directly serving the durability NFR by making partial-failure recovery cheap.

N. Deep Dive 2 — The Transcoding Pipeline

Requirement this addresses: the adaptive-playback NFR (D) — a single uploaded file must become many device/network-appropriate variants before it's genuinely watchable everywhere.

Each resolution/bitrate variant is an independent, parallelizable transcoding job off the same raw source.

Why this must be async and queue-driven, not a synchronous step in the upload request: transcoding a video into multiple quality levels is CPU-intensive and can take longer than the video's own runtime — holding an upload request open for that duration is a non-starter, the same "never put slow, heavy work on the critical path of an acknowledgment" principle used for fanout in News Feed and Chat System, just applied to a far heavier per-item workload (minutes of compute per video, versus milliseconds per fanout write).

Why parallelizing across resolutions, not just across videos: each resolution/bitrate variant is an independent transcoding job over the same raw source, so a video's variants can all be processed concurrently by different workers rather than serially — directly shortening the time-to-first-watchable-quality, which matters because the lowest-resolution variant finishing first is typically what unblocks "publish" (F's assumption) while higher resolutions continue in the background.

O. Deep Dive 3 — Adaptive Bitrate Streaming

Requirement this addresses: directly fixes V1's fatal flaw (J) — this is the actual real-world mechanism (HLS/DASH) that makes smooth playback across heterogeneous devices and networks possible at all.

How it actually works: each transcoded variant (N) is further split into short segments (typically a few seconds each); a manifest file (H) lists every available variant and its segment URLs. The video player downloads the manifest first, then continuously monitors its own current network throughput and switches which variant it's pulling segments from, in real time, between segments — dropping to a lower-bitrate variant if the network degrades mid-playback, and stepping up if it improves — all without interrupting playback or requiring a page reload.

Why this is the actual answer, not just "compress the video better": no single fixed encoding can be simultaneously right for a phone on congested cellular and a desktop on fiber, and a viewer's network conditions can change mid-playback (walking out of WiFi range) — adaptive bitrate directly solves both the heterogeneous-audience problem and the changing-conditions-mid-session problem, which a single static file, no matter how well compressed, structurally cannot.

P. Deep Dive 4 — CDN Delivery at Scale

Requirement this addresses: the global low-latency delivery NFR (D) and the ~58,000+ views/sec throughput from G — no origin server fleet can serve this volume directly, and every viewer being served from one region would add unacceptable latency for everyone else.

Why a CDN, not scaling the origin fleet directly: video segments are immutable once transcoded (a given variant's given segment never changes) — precisely the profile a CDN's edge-caching model is built for. Once a segment is cached at an edge node near a viewer, that edge node can serve it directly with no origin round-trip at all, the same cache-aside-at-the-edge principle used for redirects in URL Shortener, just applied to video segments at a much larger payload size.

The long-tail problem this creates: a small number of viral videos get requested constantly (naturally staying hot in every relevant edge cache), while the vast majority of uploaded videos are watched rarely — meaning most content is effectively "cold" at most edges most of the time, and the first viewer in a given region for a given video always pays a real origin round-trip (a cache miss) before that segment becomes locally cached. This is an accepted, bounded cost — a design choice, not a flaw — since caching every video everywhere unconditionally would be a far worse trade than occasionally eating one origin round-trip for genuinely rare content.

Q. Deep Dive 5 — Metadata & Search

Requirement this addresses: the search functional requirement (D) — a genuinely different access pattern (full-text over titles/descriptions) from the manifest/segment delivery path in O and P.

Why this is a separate system from video delivery, sharing only the video_id: the same reasoning as the Twitter/News Feed problem's search deep dive — an inverted index (Elasticsearch/OpenSearch-class) populated asynchronously off the same "video published" event already used to trigger notifications/availability, decoupled entirely from the manifest-serving path so a search-indexing lag never affects playback, and a playback issue never affects search.

R. Deep Dive 6 — View Counting at Scale

Requirement this addresses: the view-count functional requirement (D), under the approximate-counting assumption from F — 5 billion views/day (G) makes a strongly-consistent, exactly-accurate counter incremented synchronously on every single view an unnecessary and expensive goal.

Why counting can't happen synchronously on the playback path: incrementing a durable counter on every single segment/manifest request would add write load directly onto the hottest, most latency-sensitive path in the entire system — exactly backwards, mirroring the same "don't put expensive work on the highest-frequency path" lesson from URL Shortener's analytics deep dive and News Feed's fanout reasoning.

The fix: emit a lightweight ViewEvent (H) asynchronously (e.g. once playback crosses a minimum-watched threshold, to avoid counting accidental clicks) onto an event stream; a separate, async aggregation pipeline batches and periodically updates each video's displayed count. The displayed number is allowed to lag real-time by seconds-to-minutes — an explicitly accepted trade, not a bug, that removes counting entirely from the playback critical path.

S. Deep Dive 7 — Recommendations (Conceptual)

Discussed at the level this interview typically expects, per F's assumption — the conceptual shape and how it plugs into the existing pipeline, not a full ML recommendation system derivation.

Why this is a layer on top of the existing metadata/view-event streams, not a new data-collection system: the same ViewEvent stream already powering R's approximate view counts is also the natural raw signal for a recommendation model (what did this viewer watch, for how long, after watching what else) — recommendations consume the same async event stream through a different, independent aggregation/model-serving path, rather than requiring new instrumentation. This mirrors how ranking was layered onto News Feed's existing fanout/store mechanism without changing its plumbing.

T. Full Solution at Scale — Final Architecture

Everything above is the composition of M through S — no new component introduced here.

U. Design Evolution

• V1 — store and serve one raw file (J): broken for a large fraction of real devices/networks from day one, independent of scale.
• V2 — direct-to-blob-store upload via pre-signed URLs: keeps large payloads off the application tier (M).
• V3 — async, queued, parallelized transcoding into multiple variants: produces the multiple quality levels adaptive streaming needs (N).
• V4 — manifest-driven adaptive bitrate playback: fixes the actual core problem — smooth playback across heterogeneous devices/networks (O).
• V5 — CDN edge caching for segment delivery: the only way to sustain tens of thousands of views/sec globally with acceptable latency (P).
• V6 — async search indexing off the publish event: adds discovery without coupling to the delivery path (Q).
• V7 — async, threshold-based view-event aggregation: removes counting from the hot playback path entirely (R).
• V8 — recommendation pipeline consuming the same event stream: layers personalization on existing infrastructure (S).

What stays stable throughout: the video_id-keyed manifest contract from I — every version underneath changes without the player needing to change how it fetches or plays a video.

V. Consistency Model

Deliberately eventually consistent almost everywhere, matching the same overall posture as News Feed and Chat System's non-ordering-critical paths: view counts (R) lag real-time by design; search results (Q) lag publish by an indexing delay; CDN edge caches (P) can briefly serve a stale variant list until propagation completes. The one place closer-to-immediate correctness matters is upload durability itself (M) — once the platform acknowledges an upload as complete, that raw file must not be lost, since unlike a delayed view count, a lost upload is unrecoverable and directly visible to the user who created it.

W. Failure Scenarios

• A transcoding worker crashes mid-job: the job is redelivered from the queue (M-Q's standard at-least-once redelivery pattern) and reprocessed from the still-intact raw upload — no data loss, just a delay in that variant becoming available (N).
• A CDN edge node is cold for a requested segment: falls through to an origin round-trip (a cache miss), populating the edge for subsequent requests — graceful degradation to higher latency, not an outage (P).
• The view-event aggregation pipeline falls behind: isolated entirely to the displayed count — playback is completely unaffected, since it's a fully independent consumer of the event stream (R).
• An upload is interrupted mid-transfer: chunked/resumable upload (M) means only the missing chunks need to be retried, not the entire file.
• A viewer's network degrades mid-playback: this isn't a failure at all in this design — it's the exact case adaptive bitrate streaming (O) is built to handle gracefully, by switching to a lower-bitrate variant without interrupting playback.

X. Bottlenecks & Scaling Boundary

Y. Design Trade-Offs

Z. Common Bad Designs

• Store-and-serve-one-file, with no adaptive streaming at all. Delivers a visibly broken experience for a large share of real devices/networks regardless of traffic volume — the most common and most fundamental mistake on this problem.
• Synchronous transcoding held on the upload request's critical path. Ties up the acknowledgment for however long CPU-intensive multi-format encoding takes — the same async-decoupling lesson from every fanout-style problem in this document, ignored.
• Serving video directly from origin with no CDN. Cannot approach the required global throughput or latency — a design that "works" only in a single-region demo.
• Synchronous, exact view-count increments on every playback request. Adds unnecessary write load to the single hottest, most latency-sensitive path in the whole system for a number that doesn't need to be exact or instantaneous.
• Treating recommendations as a day-one requirement instead of a layer consuming an event stream that already exists for other reasons (R) — leads to over-scoping the interview into ML model design instead of the delivery architecture that carries most of the signal.

AA. Interviewer Follow-Ups

AB. Interviewer Evaluation Rubric

• Weak candidate: "store the video file, serve it on request" with no mention of device/network heterogeneity, no async processing, no CDN.
• Mid-level: knows videos need to be transcoded into multiple resolutions and that a CDN is involved, but doesn't articulate the manifest-driven adaptive-switching mechanism itself.
• Senior: proactively explains adaptive bitrate streaming (manifest + segments + real-time quality switching) as the actual mechanism, not just "compress it well"; separates the async transcoding pipeline from the synchronous upload-acknowledgment path unprompted.
• Staff+: reasons about the long-tail cold-content cache-miss cost as an accepted trade rather than a flaw; treats view counting and recommendations as approximate, async layers on the same event stream rather than new synchronous systems; can quantify the storage multiplier from multi-variant transcoding (as in G).

AC. Interview Simulation

AD. Online Solution Comparison

AE. Interview Cheat Sheet