When you hear about a graphics card or a GPU (graphics processing unit), the first thing that comes to mind is video games. But that chip is today one of the engines of artificial intelligence, cinematic rendering and scientific computing. The interesting question is not what it is for, but why it is so fast at what it does.
Sequential CPU vs parallel GPU
The essential difference is in the approach. A CPU (central processing unit) has few, very powerful cores capable of solving generic tasks in sequence. A GPU, on the other hand, packs thousands of smaller cores that execute many operations at once over a large mass of data.
An example is worth more than a spec sheet: a top-of-the-line 16-core CPU executes 16 simultaneous operations; a GPU can exceed fifteen thousand cores and launch tens of thousands of threads in a single cycle. This is achieved with an execution design called SIMD (single instruction, multiple threads): a single instruction is applied at once to an entire block of values.
The internal architecture
A GPU is not a homogeneous block. It is organized into a compute and memory hierarchy that seeks to make data flow at the highest possible speed. In simple terms, we are talking about three levels:
- Streaming Multiprocessors (SMs): the chip’s execution units, each with its own worker threads.
- Cache memory and VRAM: the intermediate and final storage, equivalent to the CPU’s cache and RAM, but designed for enormous bandwidth.
- Specialized units: circuits dedicated to specific tasks, such as the tensor cores for artificial intelligence or the ray tracing cores for lighting.
The VRAM (video memory) is one of the engineers’ big challenges: high-end GPUs exceed 700 GB/s of bandwidth, far above the system’s normal RAM. That is why the chip is designed around the memory controller and the cache hierarchy that accompanies it.
The work pipeline, from geometry to pixel
When a scene is drawn, the GPU runs a graphics pipeline. First, the vertex shader places the models’ vertices according to the camera position and perspective. Then the rasterizer converts those meshes into fragments, the minimum elements of the screen. Finally, the fragment shader calculates the color, lighting and textures of each fragment to produce the final pixel.
These shaders are today programmable: small programs that run on the GPU to achieve effects such as motion blur or dynamic shadows.
Tensor cores, the AI engine
A good part of a modern GPU’s power is no longer dedicated to graphics, but to artificial intelligence. Neural networks are, at bottom, a cascade of matrix multiplications: millions of operations at once. To speed them up, NVIDIA introduced tensor cores, silicon circuits dedicated exclusively to matrix operations.
A tensor core executes a low-precision matrix multiplication in a single cycle and accumulates results. Total power is measured in teraflops, that is, trillions of floating-point operations per second. Those teraflops are what allow training a large model in weeks instead of years, chaining together thousands of GPUs.
To join several cards, high-speed buses such as NVLink (NVIDIA) or Infinity Fabric (AMD) are used: point-to-point links thanks to which a cluster of dozens of GPUs works like a single machine with immense shared memory.
Thread management: warps and divergence
Inside, work is grouped into sets of threads that NVIDIA calls warps, of 32 elements, which are launched at once. If all 32 execute the same instruction, efficiency is maximum. If one takes a different conditional branch in the code, the warp “splits” and the branches are processed separately, with some loss of performance until they converge again. Managing that parallelism well is the key to getting the most out of the card.
Toward neural rendering
The next step is AI-powered rendering. Techniques such as NVIDIA DLSS or AMD FSR reconstruct a higher-resolution image from the calculated frame and gain fluidity without increasing the rasterization work. And neural radiance fields (NeRF) generate a complete 3D scene from just a few photographs.
The GPU has stopped being the gamer’s accessory to become the great computing center of the digital age: it renders, trains algorithms and powers data centers as well as your own computer. Knowing how it works inside is understanding one of the most decisive pieces of today’s hardware.
