Design a Video Streaming Service
YouTube is a global online video-sharing platform where users can upload, view, and share content. Netflix is a global subscription-based streaming service delivering a vast library of TV shows and movies. Both platforms require bulletproof, planet-scale streaming architectures to deliver stutter-free experiences.
How YouTube & Netflix Work (For Beginners)
Both YouTube and Netflix deliver streaming video, but their background workloads are entirely different.
YouTube is a public stage. Anyone can upload video files at any time (User-Generated Content). The system must quickly ingest, transcode, and catalog these raw uploads so viewers can discover and watch them globally in seconds.
Netflix is a curated cinema. Only administrators publish content in scheduled, high-quality batches. The engineering focus is strictly on caching and delivering a zero-lag, stutter-free playback experience to viewers globally.
The Post-Office Analogy: Imagine a global post office. An author writes a heavy book (raw video upload). The central office slices it up into small 6-second video segments (segmentation), prints them in multiple resolutions like 1080p or 4K (transcoding), and ships them to thousands of local stands worldwide (CDNs). When a reader wants to read, they fetch segment 1, and while reading, their desk automatically fetches the next few segments based on how fast they read (adaptive streaming).
In this guide, we will design a highly scalable Video on Demand (VOD) service. We will explore how to balance the write-heavy creator pipeline (uploading, transcoding, and processing) with the read-heavy distribution network (CDN edge-caching and adaptive chunk playback) to support hundreds of millions of concurrent viewers worldwide.
Functional Requirements
Must-Have Features
📤 1. Upload & Ingest
- ✓Chunked Video Upload: System must support uploading large video payloads (up to 20GB) without consuming memory limits on application servers.
- ✓Upload Pause & Resume: CRITICAL REQUIREMENT: System must support fault-tolerant uploads. If a creator's connection drops, the upload must resume from the last successful chunk without restarting.
- ✓Creator Dashboard Upload Status: Allows video publishers to monitor active upload completion and processing states.
📺 2. Playback & Streaming
- ✓Adaptive Bitrate Streaming: Transports media seamlessly using segmented streaming protocols across varying client network profiles.
- ✓Signed Playback Indexing: Grants authenticated clients geo-restricted playlist maps tied to transient IP keys.
🔍 3. Search & Discovery
- ✓Fuzzy Text Vector Search: Leverages index catalogs to lookup video records, matching descriptions and metadata instantly.
- ✓Autocomplete Type-ahead Suggestions: Generates high-speed query-matching listings in under 50ms based on historical text models.
Out of Scope & Future Nice-to-Haves
- ✕ Live-streaming pipeline ingestion (focus is strictly VOD architecture).
- ✕ Turnkey digital content licensing and copyright audit tools.
- ✕ Payment splits or recurring subscriber subscription billing structures.
- ✕ Real-time view counter aggregations (specific implementations like stream processing engines and batch state layers are deferred to the scaling/bottlenecks section).
- ✕ Content Creator Studio with in-browser clip trims.
- ✕ Real-time chat overlay for premier view countdowns.
- ✕ Interactive dynamic ad inserts mapping client sessions.
- ✕ Multi-channel playlist groupings and video collections.
Non-Functional Requirements
- ›Playback startup: < 200ms globally
- ›Edge segment retrieval: < 30ms
- ›Player buffering recovery switches in < 1s
- ›Uptime target: 99.99%
- ›Edge CDNs degrade gracefully to regional rings
- ›Decoupled architecture protects video playing during upload spikes
- ›Supports 500M Daily Active Users
- ›Accepts over 500 hours of uploaded media/min
- ›Event-driven asynchronous video transcoding
- ›Strict consistency for profile changes and uploads
- ›Eventual consistency (< 3s) for playlists and search
- ›Signed URL hashes tied to client IPs
- ›DRM encryption loops protecting licensed video content
- ›11-nines reliability via durable cloud storage
- ›Resumable uploads to prevent data loss on network drop
Back-of-the-Envelope Estimation
Throughput Math
| Metric | Calculated Target | Underlying Formula |
|---|---|---|
| Global Footprint | 500M DAUs | Active system base |
| Peak Concurrent streams | 25M users | 5% of daily active footprint at peak |
| Videos created / day | 5.4M records | 720,000 hrs uploaded / 8 min length |
| Average Query Rate | 17,400 QPS | 1.5B streams / 86,400 seconds |
| Peak Query Burst | 87,000 QPS | 5x average load multiplier |
Storage & Network Math
High-Level Design
The pipeline divides clean duties. Crucially, video transcoding is NEVER called synchronously through the API Gateway. Doing so would cause HTTP timeouts since processing a video takes minutes or hours.
When a video is sent: it hits the Upload Service which tells our storage where to put the heavy raw bytes directly. Once the storage confirms it has the file, it drops a message on the Kafka event bus. This asynchronous queue acts as a buffer and independently alerts the background conversion workers to start pulling jobs, completely decoupling heavy processing from the fast web API.
On the playback side, viewers query the Streaming Service to get a signed, customized chapters map. From that point forward, the viewer talks strictly to close-by CDN Edge nodes to fetch individual chapters, keeping our core databases fast and quiet.
Our services run statelessly. Dynamic API calls are processed via the gateway layers, and all heavy assets bypass the application servers entirely by writing directly to S3 bucket keys:
👆 Click components to trace architecture lineages
🛡️ Networking Breakdown: WAF vs. Load Balancer vs. API Gateway
In a large-scale architecture, the Web Application Firewall (WAF), Load Balancer (LB), and API Gateway (APIGW) do not represent the same server. They are distinct, decoupled infrastructure layers operating sequentially:
Located nearest to the network perimeter (often integrated at the CDN layer). Its sole job is traffic inspection: filtering SQL injection, cross-site scripting (XSS), bot scraper clusters, and Layer-7 DDoS floods before they can even touch internal services.
A highly specialized appliance (such as AWS ALB or an NGINX ring) optimized to route massive traffic. It distributes the filtered, decrypted HTTPS payloads across a cluster of API Gateway servers, acting as the primary point of failure protection.
The entrance to your internal microservice mesh. Unlike LBs, the API Gateway runs custom software logic. It coordinates downstream calls, routes paths to individual microservices (e.g., /upload vs. /search), checks request rate-limits, and communicates directly with the Auth Service.
🧩 The Core Concept: Slicing the Loaf of Bread
In system engineering, we never send a single raw video file (which could be several gigabytes) directly down a wire to a user's phone. That would cause massive buffer stalls and high data usage!
Instead, we treat a video like a loaf of bread. During the transcoding phase, we slice the video into small, 6-second segment files (like thin slices of bread) formatted in fragmented MP4 (fMP4) or TS containers.
When you hit play on YouTube or Netflix, your player fetches a map index file (called an HLS playlist or manifest). It then requests these individual 6-second slices one-by-one. If your Wi-Fi speeds slow down suddenly, the player seamlessly upshifts or downshifts the resolution of the *next* slice without crashing your viewing experience!
High-Level CAP Strategy
Strict transactional profile mapping. Relational tables guarantee absolute consistency for account state management.
Eventual consistency of metadata. Allows write speeds to scale infinitely; index delays of 1-3 seconds are visually imperceptible.
⚖️ Architectural Alternatives & Design Decisions
Data Model
User Profiles (PostgreSQL)
Used strictly for structured authentication and transaction histories requiring ACID guarantees.
Metadata DB (DynamoDB NoSQL)
Stores video catalog data. Sharded across wide-key partitions to support hundreds of millions of objects globally.
⚡ Metadata Cache (Redis)
Because a single viral video's metadata might be requested millions of times an hour, direct DB reads would bottleneck. We place a distributed Redis Cluster in front of the Metadata DB to absorb 99% of read traffic.
⚖️ Architectural Alternatives & Design Decisions
Elasticsearch Index Mapping
Transforms metadata records into high-performance search-as-you-type indices. By defining the primary search target as a search_as_you_type type field, Elasticsearch automatically breaks text inputs down into structured edge n-grams (e.g. "sy", "sys", "syst", "system").
This indexing step eliminates the need for expensive, platform-crashing database wildcard regex scans (LIKE %query%) in production. Instead, autocomplete responses resolve in O(1) time complexity directly from fast pre-tokenized memory banks.
API Design
Begins chunked multi-part session. Requires Bearer Token auth. Returns presigned URL map for TUS.
HEADERS:
Authorization: Bearer eyJhbGci...
BODY:
{
"title": "My Scale System Guide",
"file_size": 2516582400,
"mime_type": "video/mp4"
}HTTP 201 Created
{
"upload_id": "ul_01F9A...",
"video_id": "vid_01F9A...",
"chunk_size": 5242880,
"part_urls": [
{ "part_number": 1, "url": "https://s3.amazonaws.com/raw/part1?sig=..." }
]
}⚖️ Architectural Alternatives & Design Decisions
Deep Dive Subsystems
Direct Ingestion via TUS
The client negotiates with the Upload Service to generate secure, presigned S3 URLs. The client then pushes 5MB chunks directly to the S3 bucket:
VOD Upload Swimlanes
⚙️ Logical Flow of Chunked Uploads:
- Slice the File: The client splits the heavy video payload into standardized chunks (e.g., 5MB each).
- Concurrent Uploads: The player engine spawns a worker pool to upload 4 to 8 chunks simultaneously, maximizing the client's available network bandwidth.
- Direct S3 PUT: Each chunk is sent directly to the S3 bucket using its unique pre-signed URL, completely bypassing the application server's memory and CPU limits.
- Commit & Confirm: Once all part hashes (ETags) are successfully returned, the client sends a final webhook to the Upload Service, instructing it to stitch the parts together and trigger the processing pipeline.
Internal Lifecycle State machine
GPU Video Transcoding DAG (Directed Acyclic Graph)
To convert high-definition raw videos into streamable packets without blocking system threads, we split tasks into an independent **Directed Acyclic Graph (DAG)**. This ensures demuxing, multi-resolution scaling, image extraction, and watermark additions proceed in parallel pathways with isolated failure recovery:
Video Processing Directed Acyclic Graph (DAG) Subsystem
h264_nvenc) to execute these tasks concurrently. Why hardware acceleration? Dedicated physical silicon ASIC block arrays on modern GPUs process pixel conversions and video compression matrix math much faster and more efficiently than standard multi-core CPUs. Offloading raw framing computations to these dedicated circuits lowers total CPU utilization by up to 95% and reduces infrastructure costs tenfold, enabling concurrent rendering of multiple 4K/1080p target streaming ladders.⚙️ Transcoder Output Configuration Goals:
- Hardware Acceleration: Instructs the worker node to utilize the underlying NVIDIA GPU matrix for pixel processing instead of relying on CPU thread locking.
- Perceptual Bitrate Limits: Constrains the maximum visual bitrate (e.g., 5000kbps for 1080p) to balance crisp visual quality with mobile bandwidth constraints.
- Segmentation: Forces the output stream to slice exactly every 6 seconds into fragmented MP4 containers (
.m4s). - Manifest Generation: Automatically compiles the HLS
.m3u8index file pointing to the newly generated 6-second slices.
Adaptive Bitrate (ABR) Controller
Implemented directly on the client player via standard BOLABuffer-Occupancy-based Lyapunov Algorithm. An ABR logic that selects video qualities primarily based on current player buffer size. buffer-driven logic. It monitors player buffer health and switches quality classes to prevent playback stutters:
🎛️ Try It: ABR Simulator
Simulates user's active download speed (e.g., dropping in a tunnel).
How much video is pre-downloaded ahead of the current playhead.
⚙️ Buffer-Occupancy Decision Matrix:
- Emergency Fallback: If the client's forward buffer drops critically low (e.g., < 5 seconds left to play), the engine immediately downshifts to the lowest quality (e.g., 360p) to prevent a playback stall.
- Aggressive Upshift: If the client's forward buffer grows beyond a healthy threshold (e.g., > 40 seconds) and average bandwidth calculations comfortably exceed the next tier's bitrate requirements, the player seamlessly upshifts to a higher quality class (e.g., 1080p).
- Steady State: In all other scenarios, the player maintains the current quality tier to prevent jarring visual fluctuations and conserve network battery.
Adaptive Streaming: HLS & MPEG-DASH
HTTP Live Streaming (HLS) and Dynamic Adaptive Streaming over HTTP (MPEG-DASH) are the backbone protocols of modern video delivery. They solve a critical problem: How do you stream a 5GB video smoothly over fluctuating 4G/5G mobile networks without forcing the user to download the whole file first?
📄 1. The Manifest (The Menu)
Instead of a single video file, the player downloads a lightweight text file (.m3u8 for HLS, .mpd for DASH). This acts as a menu, listing all available resolutions (4K, 1080p, 720p) and providing the specific URLs for their respective video segments.
🎞️ 2. The Segments (The Slices)
The transcoder slices the video into tiny 2 to 6-second chunks (usually Fragmented MP4 .m4s files). The player downloads these chunks one by one via standard HTTP port 443. This means it trivially passes through enterprise firewalls and caches perfectly on standard CDN edge nodes.
⚖️ Architectural Alternatives & Design Decisions
Multi-Tier Cache & Origin Shield
To support millions of global users viewing heavy media files, we establish a multi-tier cache to ensure high-performance and protect our origin storage databases from crashing under heavy read loads.
- L1 - Edge CDN Caches: Thousands of points of presence (PoPs) globally positioned as physically close to the user as possible. Serves 95% of traffic.
- L2 - Origin Shield (Regional Cache): Sits directly in front of our S3 bucket. It aggregates and handles cache misses from all Edge CDNs to prevent multiple PoPs from requesting the same file from S3 simultaneously.
🛡️ The Power of Request Collapsing
Without an Origin Shield, if a new viral video is published, 50 different Edge CDN nodes worldwide might experience a cache miss for the same exact seg_01.m4s file at the same time. This means they will fire 50 simultaneous read requests to S3, incurring high costs and risking rate limits.
With an Origin Shield in place, the shield intercepts those 50 identical misses. It places 49 requests in a brief holding queue, fetches the segment from S3 just once, caches it locally, and then serves the binary to all 50 waiting Edge nodes simultaneously. This technique is known as Request Collapsing.
Security, DRM, and Content Safety
Enterprise platforms must protect copyrighted material and maintain safe community standards. We integrate these directly into the pipeline:
🔒 Token Auth (JWT)
All API endpoints validate short-lived JSON Web Tokens (JWT). Segment requests rely on signed CDN URLs tied specifically to the user's IP address and a 5-minute expiry to prevent link sharing.
🛡️ DRM Encryption
Raw files are encrypted using AES-128 standard during transcoding. Decryption keys are handed out dynamically via hardware-backed DRM frameworks (Widevine for Android/Chrome, FairPlay for Apple).
🛑 Profanity & Safety
Prior to being marked "Published", audio streams pass through an asynchronous ML transcription worker. Detected hate speech or extreme profanity flags the video for manual trust-and-safety review.
Bottlenecks & Scaling Mitigations
A creator uploading a 2GB file on a train loses cellular connection in a tunnel, halting the stream.
An account with 10M subscribers publishes a video. A 'thundering herd' occurs when millions of viewers click the video simultaneously. Because it's brand new, it isn't in the Edge CDN cache yet, meaning millions of simultaneous requests hit the origin storage, potentially crashing it.
The infinite growth of uploaded UGC videos quickly balloons S3 storage costs.
A sudden influx of creators uploading content delays transcoding times, leaving videos stuck in a pending queue.
📚 Quiz: Test Your Understanding
Check how well you learned the Video Streaming system design. 20 questions.