Video Streaming
A transcoding pipeline, adaptive bitrate, and why moving bytes close to users is the whole problem
Video is the design where bandwidth, not compute or storage, is the constraint. The numbers are large enough that the usual instincts fail.
1. Clarify and Scope
| Question | Assumption taken here |
|---|---|
| Live or on-demand? | On-demand — live changes everything |
| Who uploads? | Creators, moderated before publishing |
| Quality options? | 240p to 4K, chosen automatically |
| Offline download? | Out of scope |
| Recommendations? | Out of scope — a separate system |
2. Put Numbers On It
UPLOADS
500,000 videos/day, average 10 minutes
÷ 100,000 sec ≈ 5 uploads/sec ← small
STORAGE (this is where it hurts)
1 minute of 1080p ≈ 50 MB
10 min video ≈ 500 MB source
transcoded into 6 qualities ≈ 1.5 GB total per video
500,000 × 1.5 GB = 750 TB/day
× 365 ≈ 270 PB/year
VIEWING
100 million daily viewers × 5 videos = 500M views/day
÷ 100,000 ≈ 5,000 concurrent starts/sec
Assume 2 million concurrent streams at peak
BANDWIDTH
average stream ≈ 3 Mbps
2M × 3 Mbps = 6 Tbps ← the real problem6 Tbps is the number that decides everything
A well-provisioned datacentre connection is measured in the low hundreds of Gbps. Six terabits per second is roughly thirty datacentres' worth of egress, sustained.
You cannot serve this from your own servers. Not by scaling up, not by adding replicas.
The video bytes must be served from a CDN, from locations near the viewer. Everything else in this design is secondary to that fact.
Note the asymmetry: 5 uploads/sec against 2 million concurrent streams. Writes are trivial; reads are enormous. The upload path can be slow and careful; the playback path must be fast and cheap.
3. Define the Interface
POST /videos start an upload, returns an upload URL and video id
PUT {upload_url} the client uploads directly to object storage
GET /videos/{id} metadata plus a manifest URL
GET {manifest_url} the list of available qualities and segments
GET {segment_url} one few-second chunk of videoClients upload directly to storage, not through your servers
Routing a 500 MB upload through an application server ties up a connection for minutes and wastes bandwidth twice — in and back out.
Instead, hand the client a pre-signed URL and let it write straight to object storage. Your server handles a tiny metadata request; the bytes never touch it.
The same principle applies on the way out: your servers return URLs, and the CDN serves the bytes.
4. The Two Pipelines
5. Adaptive Bitrate
The reason video is cut into segments rather than served as one file.
Each video is encoded at several qualities, and every quality is split into segments of two to ten seconds. A manifest lists them all. The player picks a quality per segment, based on how fast the last one arrived.
manifest:
240p → segment_001.ts, segment_002.ts, ...
480p → segment_001.ts, segment_002.ts, ...
1080p → segment_001.ts, segment_002.ts, ...
Player behaviour:
segment 1 → 1080p (good connection)
segment 2 → 1080p
segment 3 → 480p (network slowed — drop quality rather than stall)
segment 4 → 720p (recovering)Why segments beat one big file
Quality can change mid-playback. A single file locks the viewer into whatever they started with — so a train entering a tunnel means buffering instead of a brief quality dip.
Seeking is cheap. Jumping to 8 minutes fetches the segments at that point, not the whole file.
Caching works properly. Segments are small, immutable and independently cacheable. A CDN can hold the popular opening segments of a million videos rather than a few whole files.
That last point matters more than it seems: most viewers watch the first 30 seconds and stop, so the early segments are far hotter than the rest.
6. Transcoding
One upload becomes many outputs, and the work parallelises well.
source.mp4
├─ split into chunks
├─ each chunk transcoded independently, in parallel
│ 240p, 360p, 480p, 720p, 1080p, 4K
├─ segment each output
└─ write manifestTranscoding is the expensive part of the whole system
Transcoding 10 minutes of video into six qualities can take longer than the video itself on a CPU. At 500,000 videos a day that is an enormous fleet.
Three things make it manageable:
Split and parallelise. Chunks transcode independently, so a 10-minute video finishes in the time of its longest chunk, not its total length.
Prioritise. Produce 480p first and publish as soon as it is ready — viewers can watch while higher qualities finish.
Transcode lazily for the long tail. Most videos are watched almost never. Generate 4K on first request rather than for every upload, and most of the work disappears.
7. What Was Traded Away
| Decision | Gained | Cost |
|---|---|---|
| CDN for all playback | The only way to serve 6 Tbps | Expensive; cache misses hit origin hard |
| Direct-to-storage upload | Servers never carry large files | Pre-signed URLs must be scoped and expire |
| Async transcoding | Uploads finish immediately | Video is not watchable for minutes |
| Segmented adaptive streaming | No stalls, cheap seeking | More files, more complex players |
| Lazy 4K generation | Large reduction in compute | First 4K viewer waits |
Follow-on Questions
Related Reading
- Caching — CDN behaviour and stampedes
- Queues and Async Work — the transcoding pipeline
- Numbers Every Design Needs — bandwidth arithmetic
- URL Shortener — the same async-counting trick