Malloc does not call the kernel for every byte
When a C or C++ program asks for memory with malloc(), most people picture the operating system handing out fresh pages every time. The truth is duller and far more efficient: malloc is not a system call, it is a library function. On Linux with GNU libc, the one answering is ptmalloc2, glibc’s allocator, which decides whether it can serve the request from a reserve it already holds inside the process or whether it must, only then, ask the kernel for more land via brk() or mmap().
That reserve is the whole point: the allocator keeps its own heap and carves it into chunks. The kernel only shows up when the allocator runs out of room, and when it does it hands out large blocks, not the 16 bytes you just asked for.
The header nobody sees
Every block malloc returns carries a hidden header (a chunk header) of 16 bytes on 64-bit architectures: the block size plus a few status bits such as PREV_INUSE, which says whether the previous chunk is in use. That is why asking for 8 bytes never really costs 8 bytes, and why every returned address is 16-byte aligned. The design trick is elegant: the next chunk’s header reuses the current chunk’s last field while that chunk is allocated, so not a single byte of metadata is wasted.
When you call free(), the block does not go back to the operating system: it goes back to an internal list held by the allocator (the bins: fastbins for small sizes, smallbins, the unsorted bin and largebins), sorted by size, where it waits for the next request. Allocating and freeing 100,000 times may not cost a single kernel call.
tcache: the per-thread micro-cache that changed the rules
Since glibc 2.26 there is tcache (thread cache): each thread owns a cache of 64 buckets holding up to 7 chunks each, with no need to grab the heap’s global lock. The result is a malloc/free pair of a few nanoseconds in the hot path, and far less contention when dozens of threads allocate at once.
It is also a minefield for security: if an attacker corrupts a pointer inside tcache, the next malloc can return an arbitrary address. Since glibc 2.32 this is mitigated with safe-linking, which stores list pointers obfuscated with the address of their own slot.
Fragmentation: why your process grows but never shrinks
Two different effects, easily confused. Internal fragmentation is the space wasted inside a chunk: you asked for 33 bytes, the allocator rounds up to 48 and nobody uses the remaining 15. External fragmentation is worse: free memory exists, but scattered in holes too small for a big request, so the process asks the kernel for new land even though plenty of memory is idle. And because freed chunks stay in those lists, the process RSS (Resident Set Size, the physical memory it truly occupies) tends to climb and not come back down.
From the heap to the silicon: pages, page faults and THP
When the allocator asks for 4 MB with mmap(), the kernel does not hand over physical memory: it hands over a range of virtual addresses. Memory actually appears when the program touches each page, triggering a minor page fault that the kernel fault handler resolves by finding a free physical frame (from the buddy allocator for 4 KB pages, and from the slab allocator for small kernel objects) and filling in the page tables. If the page was not in the TLB, the cache that translates addresses, those tables must also be walked: that is why the first touch of new memory is noticeably more expensive than the second.
The modern trick to cut that bill is Transparent Huge Pages: the kernel groups 512 pages of 4 KB into a single 2 MB page, with one TLB entry and far fewer faults. The price is wasted memory when a program only touches one byte out of every 2 MB block, which is why databases are often told to turn them off.
Alternatives: jemalloc, tcmalloc and mimalloc
glibc is not the only allocator. Servers commonly run jemalloc (born in FreeBSD, adopted by Redis, Cassandra and Firefox) or tcmalloc (Google’s), which organise memory into per-thread arenas and fixed size classes, cutting both locks and fragmentation. mimalloc, from Microsoft, uses dedicated pages and sharded per-thread free lists; and hardened_malloc, the one in GrapheneOS, puts security first: it separates metadata behind guard pages, randomises block placement and catches overflows, at the cost of speed and a good deal more memory.
All of this is observable rather than guesswork: /proc/PID/smaps shows every anonymous mapping, mallinfo2() summarises ptmalloc’s internal state, and tools such as Heaptrack or Valgrind Massif draw who allocates what. In containers it also pays to cap MALLOC_ARENA_MAX: the default of eight arenas per core can inflate a multithreaded process’s memory to surprising levels.
All of the above explains an uncomfortable sentence worth internalising: freeing memory in your program does not give it back to the operating system. The allocator is a patient middleman that would rather keep than return, and that behaviour, invisible almost always, surfaces exactly when the server runs out of memory at three in the morning.
