You open your favourite series in 4K and barely notice a detail bordering on the impossible: the video you are watching travels down your fibre at about 15–25 megabits per second (Mbps). Run the numbers and things get absurd: without compression, that same stream would demand around 12,000 Mbps — more than 5 terabytes (TB) per hour. The fact that your connection can handle the whole film is the work of video compression, one of the most sophisticated pieces of engineering you use daily without realising it.
The starting problem: too much data
A 4K frame is 3,840 pixels wide by 2,160 tall, and each pixel needs at least 3 bytes to define its colour (red, green and blue — the RGB model). Result: about 24.9 megabytes (MB) per image. At 60 frames per second, one second of uncompressed 4K video weighs ~1.5 GB: more than 60 high-resolution photos. An hour-long episode would be about 5.4 TB. No network or disk can cope with that.
Codecs solve the bottleneck by exploiting a property of video: redundancy. There are three kinds, and each is attacked with a different technique.
Spatial redundancy: the frame as an image
Within a single frame, neighbouring pixels almost never change abruptly: a blue sky has millions of nearly identical pixels. Instead of storing every pixel, the codec splits the image into blocks and stores a compact approximation of each. The key piece is the discrete cosine transform (DCT), which converts a block’s values into frequency coefficients: fine detail (high frequencies) is rounded more aggressively than uniform areas, because the human eye barely notices it. That rounding is the “lossy” part of compression, and its intensity is controlled by the bitrate.
Temporal redundancy: frames that only store differences
Between two consecutive frames almost everything repeats: only a character or the camera moves. Codecs distinguish three types of frames:
- I-frames (intra): compressed on their own, like a JPEG. They are the video’s “anchors”, the heaviest, and appear at intervals (for example every 2 seconds) to allow seeking and error recovery.
- P-frames (predicted): store only the difference from the previous frame.
- B-frames (bidirectional): interpolate from past and future frames. They are the most efficient, but force frames to be reordered, adding latency; that is why live video uses few or none.
The trick behind P and B frames is motion compensation: the codec splits the image into blocks and looks, in the reference frame, for where each block has moved. Instead of sending the block’s content, it sends a motion vector (two numbers: horizontal and vertical displacement) plus the small residual error. Most of a modern video is, literally, instructions like “move this block two pixels to the right”.
The final touch: entropy coding
Everything above produces symbols (vectors, coefficients, errors) that still occupy bits. Entropy coding — in modern codecs, CABAC (Context-Adaptive Binary Arithmetic Coding) — assigns shorter codes to the most frequent symbols, just as Morse uses a single signal for “e”. At the end, the codec produces a bitstream that hardware decoders reconstruct in real time.
The codec war: from H.264 to AV1
The efficiency of all this is measured in how much bitrate is needed for a given quality. The oldest and most universal standard is H.264/AVC (2003): absolutely everything decodes it. H.265/HEVC (2013) improved it by around 50%, but its patent maze slowed adoption outside 4K Blu-ray and streaming from Apple or Netflix on certain platforms.
Facing the licences, the industry formed the Alliance for Open Media (Google, Mozilla, Netflix, Amazon, Microsoft and, since 2018, Apple) and created AV1, a royalty-free codec that performs ~30% better than HEVC. The price: encoding AV1 in software is extremely slow — a film can take hours or days — although hardware decoding is already in phones, TVs and graphics cards, and platforms like YouTube or Netflix use it massively. AV1 also gave its name to the AVIF image format.
The latest link, VVC/H.266 (2020), compresses another 30–50% more, but is again held back by patents; the alliance is already working on AV2. Meanwhile, neural codecs based on deep learning promise to outperform them all, and AI upscalers (such as DLSS in graphics cards applied to video) attack the problem from the other side: transmit less resolution and reconstruct it on the device.
Next time you watch a film
Next time you play a 4K video, remember the paradox: that second of video fits in less space than a single 4K photo because almost all the data was predictable. The codec is the brain that finds the redundancy, discards it and reconstructs it on your screen, in real time and without you noticing the difference. The magic is not in the screen: it is in the mathematics behind every block.
