Every time you load a website with real traffic, an invisible component is deciding which server should receive your request. It is not magic and not a secret jury: it is the load balancer, the entry point that distributes the work so that no single server gets saturated.
Why you need a traffic light
When a service grows, a single server is not enough: CPU, memory, and bandwidth have physical limits. The solution is to replicate the service across several identical servers, a cluster (a group of machines that cooperate as if they were one). But that introduces a new problem: if each user points directly at a different server, some get overloaded while others sit idle. The load balancer sits in front of them all and acts as a single point of entry (a reverse proxy, i.e. an intermediary that receives requests and forwards them to the internal servers).
Two levels of distribution: L4 and L7
The load balancer works at two layers of the OSI model (the reference framework that splits network communication into seven layers). At layer 4 (transport) it decides only by looking at the IP address and port: it knows who the request is for, but not what it contains. It is extremely fast, because it barely touches the packet, and ideal for raw traffic such as databases or VoIP (voice over IP). At layer 7 (application) the balancer inspects the full content, for example the URL or the HTTP headers (the metadata that travels with every web request), and can route by path, domain, or even by user. This is the one used by proxies such as nginx or HAProxy to serve a frontend and an API on the same infrastructure.
The decision algorithms
The key question is: which server do I send each request to? There are several strategies. The simplest is round-robin: cyclic turns, one for each server in order, like dealing cards around a table. Least connections sends the request to the server with the fewest active connections at that moment, which adapts better when each request takes a different amount of time. Consistent hashing computes a number from the client’s IP or a key and always assigns that user to the same server; that lets you reuse a cache or a session without losing it between requests.
Health, sessions, and security
A load balancer does not distribute blindly: it runs health checks (periodic health probes) that send a test request to each server and, if one fails to respond or returns an error, it temporarily removes it from the pool. It also manages persistent sessions (sticky sessions): once a user has logged in on a server, their subsequent requests should return to that same server, or they would have to re-authenticate on every click. And it plays a security role: as the only point exposed to the internet, that is where TLS (the encryption that protects HTTPS traffic) ends, so the internal servers can talk to each other unencrypted, on a private network.
From hardware to software and the cloud
In the past, load balancing was done with dedicated hardware (specialized network appliances from vendors like F5 or Citrix), expensive but extremely high-performing. Today software dominates: nginx, HAProxy, or Kubernetes’ load balancer, the container orchestration platform that also scales the number of replicas according to load. In the cloud, managed services such as AWS Application Load Balancer or Google Cloud Load Balancing offer the same concept without you having to administer the infrastructure.
The next time a website survives a sale with thousands of simultaneous visitors, you already know who does the dirty work: a silent traffic light that, with a simple algorithm, decides every millisecond which server should receive your request so that it never goes down.
