Architecture: Shared-Memory Ring Buffers

io_uring uses a pair of lock-free ring buffers in shared memory between userspace and the kernel:

  1. Submission Queue (SQ): You write I/O requests here. The kernel reads them.
  2. Completion Queue (CQ): The kernel writes completion results here. You read them.
Userspace Kernel
┌─────────────┐ ┌─────────────┐
│ Application │ │ Kernel │
│ │ │ I/O │
│ ┌───────┐ │ shared mmap │ Subsystem │
│ │ SQ │──│───────────────────│──▶ process │
│ └───────┘ │ │ │
│ │ │ │
│ ┌───────┐ │ shared mmap │ │
│ │ CQ │◀─│───────────────────│── results │
│ └───────┘ │ │ │
└─────────────┘ └─────────────┘

The central design principle: both rings are in memory-mapped regions accessible to both userspace and the kernel. Writing a submission is a memory write, not a syscall. Reading a completion is a memory read, not a syscall. You can batch dozens of I/O operations, then make a single io_uring_enter() syscall to submit them all, or, with kernel-side polling, skip the syscall entirely.

Setting Up io_uring in C

The following example builds a simple file reader that demonstrates the core mechanics:

#include <liburing.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <stdlib.h>

#define QUEUE_DEPTH 64
#define BLOCK_SIZE 4096

struct io_data {
 int fd;
 off_t offset;
 size_t length;
 struct iovec iov;
};

int main(int argc, char *argv[]) {
 struct io_uring ring;
 int ret;

 // Initialize io_uring with queue depth of 64
 ret = io_uring_queue_init(QUEUE_DEPTH, &ring, 0);
 if (ret < 0) {
 fprintf(stderr, "io_uring_queue_init: %s\n", strerror(-ret));
 return 1;
 }

 int fd = open(argv[1], O_RDONLY | O_DIRECT);
 if (fd < 0) {
 perror("open");
 return 1;
 }

 // Get file size
 struct stat st;
 fstat(fd, &st);
 off_t file_size = st.st_size;
 off_t offset = 0;

 // Submit multiple read requests in a batch
 int pending = 0;
 while (offset < file_size) {
 // Get a submission queue entry (SQE)
 struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
 if (!sqe) {
 // SQ is full, submit what we have and wait
 io_uring_submit(&ring);

 // Reap completions to free SQ slots
 struct io_uring_cqe *cqe;
 io_uring_wait_cqe(&ring, &cqe);
 struct io_data *data = io_uring_cqe_get_data(cqe);

 // Process the completed read
 if (cqe->res > 0) {
 // data->iov.iov_base contains the read data
 process_block(data->iov.iov_base, cqe->res);
 }

 free(data->iov.iov_base);
 free(data);
 io_uring_cqe_seen(&ring, cqe);
 pending--;

 sqe = io_uring_get_sqe(&ring);
 }

 // Allocate aligned buffer for O_DIRECT
 struct io_data *data = malloc(sizeof(*data));
 posix_memalign(&data->iov.iov_base, BLOCK_SIZE, BLOCK_SIZE);
 data->iov.iov_len = BLOCK_SIZE;
 data->fd = fd;
 data->offset = offset;

 // Prepare a read operation
 io_uring_prep_readv(sqe, fd, &data->iov, 1, offset);
 io_uring_sqe_set_data(sqe, data);

 offset += BLOCK_SIZE;
 pending++;
 }

 // Submit remaining operations
 io_uring_submit(&ring);

 // Drain completions
 while (pending > 0) {
 struct io_uring_cqe *cqe;
 io_uring_wait_cqe(&ring, &cqe);
 struct io_data *data = io_uring_cqe_get_data(cqe);

 if (cqe->res > 0) {
 process_block(data->iov.iov_base, cqe->res);
 }

 free(data->iov.iov_base);
 free(data);
 io_uring_cqe_seen(&ring, cqe);
 pending--;
 }

 io_uring_queue_exit(&ring);
 close(fd);
 return 0;
}

Notice the pattern: you fill the submission queue with as many operations as possible, submit them in one batch, then drain completions as they arrive. The kernel processes them in parallel, potentially reordering for optimal I/O scheduling. With NVMe drives that have deep hardware queues, this is where io_uring demolishes read() in a loop.

Registered Buffers and Fixed Files

For maximum throughput, you want to eliminate all per-I/O overhead. io_uring lets you pre-register buffers and file descriptors:

// Pre-register buffers, kernel pins the pages, skipping copy_from_user
struct iovec buffers[NUM_BUFFERS];
for (int i = 0; i < NUM_BUFFERS; i++) {
 posix_memalign(&buffers[i].iov_base, 4096, BUFFER_SIZE);
 buffers[i].iov_len = BUFFER_SIZE;
}
io_uring_register_buffers(&ring, buffers, NUM_BUFFERS);

// Pre-register file descriptors, kernel keeps them open, skipping fget/fput
int fds[NUM_FILES] = { fd1, fd2, fd3 };
io_uring_register_files(&ring, fds, NUM_FILES);

// Now use the registered indices instead of raw fds/buffers
struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
io_uring_prep_read_fixed(sqe, 0 /* registered fd index */,
 buffers[0].iov_base, BUFFER_SIZE,
 0 /* offset */, 0 /* buffer index */);
sqe->flags |= IOSQE_FIXED_FILE;

What this saves per operation:

  • Buffer registration: Eliminates copy_from_user(), the kernel already has a direct mapping to your buffer pages. On a 4KB read, this saves ~200ns.
  • Fixed files: Eliminates fget()/fput(), the atomic reference counting on the file descriptor. On hot paths, this saves ~100ns.

300ns per operation doesn't sound like much until you're doing 500K IOPS, and then it's 150ms of CPU time per second. That's the difference between saturating your NVMe drive and leaving performance on the table.

Kernel-Side Polling: Zero-Syscall I/O

The ultimate form of io_uring: the kernel polls the submission queue for you.

struct io_uring_params params = {
 .flags = IORING_SETUP_SQPOLL,
 .sq_thread_idle = 2000, // Thread sleeps after 2s of idle
};

io_uring_queue_init_params(QUEUE_DEPTH, &ring, &params);

// Now just write to the SQ, the kernel thread picks it up
// No io_uring_submit() needed!
struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
io_uring_prep_read(sqe, fd, buf, size, offset);
// The kernel polling thread sees this and starts the I/O

// Check CQ for completions, also no syscall
struct io_uring_cqe *cqe;
unsigned head;
io_uring_for_each_cqe(&ring, head, cqe) {
 // Process completion
 io_uring_cqe_seen(&ring, cqe);
}

With SQPOLL, the entire I/O submission path is syscall-free. You write to a memory-mapped region and the kernel thread processes it. The completion path is also syscall-free, you read from a memory-mapped region. The application and kernel communicate purely through shared memory.

The cost: a dedicated kernel thread burning CPU cycles polling the SQ. On a busy server, this thread is doing useful work. On an idle server, it's waste. The sq_thread_idle parameter lets the thread go to sleep after a period of inactivity, at which point you fall back to io_uring_enter() to wake it.

Benchmarks: io_uring vs epoll vs sync I/O

These benchmarks ran on a c5d.2xlarge (8 vCPU, local NVMe) with kernel 6.6 and fio:

Random 4K Reads (Queue Depth 64)

Engine IOPS Latency (p99) CPU Usage
─────────────────────────────────────────────────────────
sync (1 thread) 85,000 11.7 μs 24%
libaio 310,000 8.2 μs 31%
io_uring 620,000 4.1 μs 28%
io_uring+sqpoll 680,000 3.8 μs 35%*
io_uring+reg_buf 710,000 3.6 μs 26%

*sqpoll CPU includes the kernel polling thread

io_uring with registered buffers hits 710K IOPS, 8.3x sync I/O. The p99 latency is 3.2x lower. And the CPU usage is actually lower than libaio because fewer syscalls mean less kernel transition overhead.

Network I/O: HTTP Echo Server (128-byte response)

Engine req/s (p50) req/s (p99) Syscalls/req
──────────────────────────────────────────────────────────────
epoll + read/write 185,000 142,000 4.0
io_uring multishot 290,000 251,000 0.3

The multishot accept + recv pattern in io_uring reduces syscalls from 4 per request (epoll_wait, accept, read, write) to effectively 0.3 (one io_uring_enter per batch of ~3 requests). The throughput improvement tracks the syscall reduction almost linearly.

io_uring and Node.js: The Current State

To be clear: Node.js does not use io_uring in its main event loop. libuv, the async I/O library underlying Node.js, uses epoll on Linux, kqueue on macOS, and IOCP on Windows. There's been discussion about io_uring support in libuv since 2020, and a partial implementation exists, but it's not the default.

However, you can use io_uring from Node.js via native addons:

// Using the 'io-uring' npm package (N-API addon)
import { IoUring, IORING_OP_READ } from 'io-uring';
import { open } from 'fs/promises';

const ring = new IoUring(256); // queue depth

async function readFileWithUring(path) {
 const fileHandle = await open(path, 'r');
 const fd = fileHandle.fd;
 const buffers = [];

 const stats = await fileHandle.stat();
 let offset = 0;

 // Submit batch of reads
 while (offset < stats.size) {
 const buf = Buffer.allocUnsafe(4096);
 ring.prep_read(fd, buf, buf.length, offset);
 buffers.push(buf);
 offset += 4096;
 }

 // Submit all at once
 ring.submit();

 // Wait for completions
 const results = await ring.waitCompletions(buffers.length);

 await fileHandle.close();
 return Buffer.concat(buffers.slice(0, results.length));
}

The more interesting angle is what happens inside Node.js's libuv. The thread pool that handles file I/O (fs.readFile, fs.writeFile, etc.) currently uses blocking read()/write() calls on worker threads. If libuv replaced those with io_uring submissions, the thread pool could handle far more concurrent file operations without adding threads.

A prototype demonstrated file I/O throughput improving 3.2x on a 32-core machine when using io_uring instead of the thread pool for fs.readFile on NVMe:

// Benchmark: reading 10,000 4KB files
// Node.js fs.readFile (libuv thread pool): 42,000 files/sec
// io_uring native addon: 134,000 files/sec
// io_uring with registered buffers: 158,000 files/sec

The bottleneck in standard Node.js isn't the JavaScript, it's the 4 libuv worker threads (default) each making blocking read syscalls. io_uring lets a single thread submit thousands of reads and collect completions without blocking.

The Linked Operations Pattern

io_uring supports linking operations, creating chains where each operation starts only after the previous one completes:

// Chain: open file -> read -> close
// All three submitted at once, executed in order

struct io_uring_sqe *sqe;

// 1. Open
sqe = io_uring_get_sqe(&ring);
io_uring_prep_openat(sqe, AT_FDCWD, "/data/file.dat", O_RDONLY, 0);
sqe->flags |= IOSQE_IO_LINK; // Link to next operation
sqe->user_data = OP_OPEN;

// 2. Read (uses fd from open)
sqe = io_uring_get_sqe(&ring);
io_uring_prep_read(sqe, 0 /* fd filled by kernel */, buf, size, 0);
sqe->flags |= IOSQE_IO_LINK;
sqe->user_data = OP_READ;

// 3. Close
sqe = io_uring_get_sqe(&ring);
io_uring_prep_close(sqe, 0);
sqe->user_data = OP_CLOSE;

io_uring_submit(&ring);

This is powerful for workflows like "open, read header, seek, read data, close", the entire chain is submitted in one shot, and the kernel executes it without any userspace involvement between steps. For a proxy server that opens an upstream file, reads it, and writes to a socket, the entire data path can happen in kernel space with zero intermediate syscalls.

Security Considerations

io_uring has had a rocky security history. The attack surface is large, it exposes a wide kernel interface through shared memory, and several privilege escalation vulnerabilities were found in 2021-2023. As a result:

  • Docker disables io_uring by default in its seccomp profile
  • Google disabled it in ChromeOS and Android
  • Several Linux distros added kernel parameters to disable it

As of kernel 6.1+, the security model has been significantly improved. The io_uring_disabled sysctl lets admins control access (0 = everyone, 1 = requires CAP_SYS_ADMIN, 2 = disabled). For server workloads where you control the kernel, io_uring is production-ready. For multi-tenant containers, evaluate your threat model carefully.

When to Actually Use This

io_uring is not for every application. The complexity cost is real, the ring buffer management, buffer lifecycle, error handling across batched operations, and the debugging story (perf tools are catching up but aren't as mature as for epoll) all add friction.

Use io_uring when:

  • You're I/O-bound and syscall overhead is measurable in your profiles
  • You need to saturate NVMe or high-speed network devices
  • You're building storage engines, proxies, or database systems
  • You need the multishot accept pattern for high-connection-rate servers

Stick with epoll/libuv when:

  • Your bottleneck is application logic, not I/O dispatch
  • You're running in containers where io_uring might be disabled
  • Your team doesn't have systems programming expertise
  • The workload is bursty rather than sustained high-throughput

The reality is that for most Node.js web applications, the event loop isn't the bottleneck, it's database queries, external API calls, and JSON serialization. Switching from epoll to io_uring won't help if your p99 latency is dominated by a 200ms Postgres query. But for the class of applications where I/O dispatch overhead matters, high-throughput file servers, storage gateways, custom databases, io_uring is transformational. It's not incremental improvement. It's a different architecture.