Home / Software y Cloud / How Linux stopped asking the kernel for permission on every disk read

How Linux stopped asking the kernel for permission on every disk read

For decades, talking to a disk from a program meant one request after another: the process called into the kernel, the kernel did the work, the process waited and called again. Every one of those system calls costs a CPU mode switch, and on a server pushing millions of operations per second that toll becomes the real bottleneck. io_uring, the asynchronous I/O interface Jens Axboe added to the Linux kernel in version 5.1 (2019), changed the model: instead of asking the kernel for permission on every operation, the program and the kernel share a pair of queues in memory and notify each other through them.

What it costs to cross the border

A classic system call like read() forces the CPU from user mode into kernel mode, validates arguments, runs the device driver and comes back. If the data is not in cache, the process goes to sleep and the scheduler puts another task in its place: two context switches per operation. With thousands of open files and microsecond latencies, adding tens of nanoseconds of toll to every read wrecks any performance budget. Traditional POSIX AIO did not help either: on Linux it ended up implemented in user space with threads, so there were still threads, stacks and scheduling behind every operation.

The idea behind io_uring is easy to state: if the work is already described in shared memory, the kernel does not need to ask anything again. The program prepares the operations, rings a bell and moves on; the results show up in the other queue.

Two rings, one bell and shared memory

The heart of io_uring is two circular rings mapped into the process address space with mmap. Into the submission queue (SQ) the program writes operation descriptors, the SQE: a small block holding the opcode, the file descriptor, the buffer pointer, the offset and a flags field. Into the completion queue (CQ) the kernel drops the results, the CQE: which operation finished, how many bytes moved and the error code. Both queues have a head and a tail index that each side reads and writes with memory barriers, so neither needs a system call to know whether the other has advanced.

The only mandatory crossing is the bell: io_uring_enter() tells the kernel there are new operations in the SQ and, if the program wants, how many completions to wait for before returning control. In SQPOLL mode even that is not needed: a kernel thread sits polling the submission ring, so the program can dispatch I/O without a single system call. The price is a dedicated CPU spinning idle; in exchange, latency drops to values that used to require hand-rolled busy polling.

Fewer copies: fixed descriptors and buffers

So far we saved system calls, but the data still travels. io_uring attacks that front too with registration: the program can pin a set of buffers (IORING_REGISTER_BUFFERS) and file descriptors (IORING_REGISTER_FILES) in advance. Registering them makes the kernel translate their pages once and keep the table; from then on SQEs no longer carry real pointers and descriptors, only indexes, which removes the repeated validation and address translation on every operation. Fixed files also let it operate on descriptors without keeping a reference per operation open, which is valuable on servers handling ten thousand connections.

The highest-impact modifiers build on that substrate: IORING_OP_READ_FIXED and WRITE_FIXED write straight into the registered buffer, and the zero-copy flags (IORING_FEAT_SEND_ZC) let the kernel send from the process pages without copying them into the socket buffer. The copy does not vanish by magic — the data still has to reach the device — but unnecessary duplication between layers is avoided.

What you can ask for on the same ring

io_uring is not just file reads and writes. The opcode catalogue includes OPENAT, STATX, FSYNC, ACCEPT, CONNECT, SEND, RECV, POLL_ADD, TIMEOUT and even SPLICE, so a network server can do all of its work — accept, read, write, time out and close — inside the same mechanism. Two details change the game: operation chaining (IOSQE_IO_LINK), which makes one operation depend on the success of the previous one without returning to user space, and multishot operations, which stay alive and produce several completions, for instance a RECV serving every read on a connection without rearming it each time. With completion polling (IORING_SETUP_IOPOLL) interrupts can even be avoided, leaving a loop that only watches the CQ.

Who uses it, and why the database world went first

RocksDB (and with it much of key-value storage), ScyllaDB, Redpanda and fio benchmarks adopted it early, because they are workloads with a huge number of small, parallel operations where the per-syscall toll dominates. PostgreSQL has spent years carrying asynchronous I/O patches built on io_uring to close in on the I/O that InnoDB already had. In language land, Rust async runtimes wrap it through liburing (glommio, tokio-uring), and C itself has that thin layer that saves you from writing the ring protocol by hand. The reason is always the same: turning “one operation, one system call” into “n operations, one system call”.

An awkward detail: the border is very wide

All that power has a flip side. By sharing structures with the kernel and allowing arbitrary operations to be chained, io_uring suddenly widened the surface where an access control check can fail, and that turned into a run of named vulnerabilities between 2022 and 2024. Google blocked its use from user space in ChromeOS and on Android with seccomp, browsers and several container services have disabled or filtered it, and some hardened distributions ship it off by default. The debate is still open: for a self-owned server where you control everything running on top, the gain is large; in a multi-user environment, it is one more interface to watch. The kernel, meanwhile, has been closing holes and cutting the cost of operations that used to demand more validation.

What it means in practice

In a world where NVMe drives settle a read in less time than a CPU mode switch takes, spending half the budget talking to the kernel was an obvious waste. io_uring adds no new algorithm: it adds a mechanism, a shared-memory contract that lets work be described in batches and results be discovered without crossing into the kernel more than strictly necessary. It is the same move as a send buffer on a network card: the longer the queue of work handed over at once, the less the handover itself costs. Next time a server holds tens of thousands of connections without breaking a sweat, chances are there are two rings of memory underneath and a bell that almost never rings.