In a large data center, disks fail every single day. It is not a catastrophe: it is routine. And yet when you open your photo folder in the cloud, everything is there. The usual explanation is that there are copies, but copying is the most expensive way of not losing data: keeping three replicas of a petabyte means three petabytes of disks. What serious systems do is something else entirely. They split every file into fragments and add parity fragments computed with algebra. When a disk dies, nobody reaches for a copy: they solve an equation.
That trick has a name: erasure coding, and its most widespread form is Reed-Solomon coding, published in 1960 by Irving Reed and Gustave Solomon. It lives inside CDs, QR codes, digital television and the probes that send pictures back from Jupiter. And for years now, it also lives in the heart of cloud storage.
Splitting and computing beats copying
Let us compare two ways of storing a petabyte of user data. The first, triple replication, is what any simple system does: three identical copies on three different servers. That is 200% overhead, so a petabyte of user data becomes three petabytes of disks.
The second is 12+4 coding. The file is split into 12 data fragments and 4 parity fragments are computed from them. That tolerates four simultaneous failures and drops overhead to 33%: the same petabyte now occupies 1.33 petabytes raw. Same data, same resilience, less than half the disks.
This is not laboratory theory. Facebook built its warm object store, the f4 system, around a Reed-Solomon (10,4) scheme with 1.4x overhead. Google moved from the three replicas of the old GFS to a (6,3) scheme in Colossus. Hadoop 3 ships native erasure coding in HDFS with a default policy of RS-6-3-1024k (6 data fragments, 3 parity fragments, 1 MB stripes). Ceph lets you set the number of data and parity fragments in the pool profile, and Azure uses its own scheme called LRC 12+2+2. The important detail: none of those fragments is a copy of the file. None of them, on its own, contains your photo.
The algebra underneath: a byte is not just any number
To understand how something that exists nowhere can be rebuilt, you have to drop one level down. Reed-Solomon does not treat data as ordinary integers but as elements of a finite field: GF(28), the Galois field with 256 elements. Every byte is a polynomial of degree below 8 with coefficients of 0 or 1, and the operations are redefined.
Addition is a bitwise XOR, with no carries. Multiplication is a polynomial product reduced modulo an irreducible polynomial, typically x8+x4+x3+x2+1 (0x11D in hexadecimal). The result always fits in 8 bits. What looks like gratuitous complexity is exactly what makes the algebra work on bytes, reversibly and without any loss.
With that arithmetic you build a generator matrix of k rows and n columns, usually a Vandermonde or Cauchy matrix. The n fragments come from multiplying that matrix by the k data fragments. The key property is called MDS, maximum distance separable: any subset of k fragments out of the n in existence is enough to rebuild the original data. The decoder picks k surviving fragments, assembles a square k×k submatrix, inverts it and solves. When failures are localised the system knows exactly which fragments are missing and needs precisely m parity elements to repair them, where m = n − k. If there are also silent errors at unknown positions, the correction capacity is halved.
Multiplying in GF(28) is cheap if you ask the CPU nicely
For decades that arithmetic was done with logarithm and antilogarithm tables: 256 precomputed entries that turned multiplications into index additions. It works, but it is slow and full of random memory accesses.
Today it is done with dedicated silicon. PCLMULQDQ computes carry-less polynomial products (the binary equivalent of schoolbook multiplication without carries), AVX-512 does several blocks at once, and GFNI adds Galois field operations directly to the instruction set. Libraries such as Intel ISA-L or jerasure implement the encoder and decoder on top of those instructions and reach several gigabytes per second per core. Encoding a file now costs less than compressing or encrypting it, which is why erasure coding stopped being a luxury reserved for cold storage.
What reads cost and what repairs cost
The price is not in the CPU, it is in the network. A read that touches a 12+4 coded file has to gather 12 fragments from 12 different disks: fetch them all, wait for the slowest one and reassemble them in memory before decoding. That inflates tail latency and punishes small files, so hybrid systems keep hot data replicated and reserve coding for warm and cold tiers. HDFS in fact recommends its erasure-coded policies explicitly for cold data only.
Repairing is even less comfortable. In a classic Reed-Solomon scheme, rebuilding a single lost fragment requires reading k fragments: twelve reads to repair one. Multiply that by millions of disks and network amplification becomes an engineering problem in its own right. That is where local reconstruction codes come from. Azure’s 12+2+2 scheme groups data into two local groups with their own parity: if one fragment dies, reading two is enough to rebuild it, and only multiple simultaneous failures force you to fall back on the two global parities. Same space overhead, a fraction of the repair traffic.
The same reasoning explains the durability figures vendors advertise. Azure local replication promises eleven nines (99.999999999%); geo-redundant storage reaches sixteen. Backblaze vaults arrange twenty groups of disks with a 17+3 scheme that survives the loss of three entire groups. And all those numbers rest on a debatable assumption: that failures are independent of each other.
What algebra does not fix
Erasure coding protects against the loss of fragments whose position is known. It does not protect against silent corruption: if a bit flips inside a fragment that is still there, the decoder will happily take it as valid. That is fought with per-block checksums and scrubbing, the background process that re-reads and verifies data. Neither does it protect against correlated failures: three disks can die at once because they share a rack, a power supply or a buggy firmware. That is why serious schemes spread fragments across failure domains, and why Ceph talks about CRUSH rules while Azure talks about zones. And of course it does not protect against deletion: for that you need versioning and object lock.
There is one last factor pushing everything in the same direction in 2026: disks are enormous. With drives above 30 TB, rebuilding a failed disk takes days, and the vulnerability window stays open for all of it. The longer the repair takes, the more it pays to spread the data around and to have local parities that rebuild fast and read little.
From the outside, the cloud looks like magic: a disk dies and nobody loses a photo. From the inside, it is the same mathematics that keeps a scratched CD playing, applied to bytes instead of bits, and executed at gigabytes per second on processors that dedicate entire instructions to multiplying polynomials. A piece of 1960 algebra holding up the least glamorous and most important part of the internet.
