TCP vs QUIC vs HTTP/3: Kernel Bypass, Zero-Copy, and Latency Optimization

The testing setup

Before examining the numbers, it is worth being transparent about the hardware, too many "benchmark" posts are run on a random EC2 instance with noisy neighbors.

• Server: Bare metal, AMD EPYC 9454 (48C/96T), 256GB DDR5-4800
• NIC: Mellanox ConnectX-7 (100GbE), dual-port
• Client: Same hardware, direct connection via DAC cable (no switch)
• OS: Ubuntu 24.04 LTS, kernel 6.8.0
• DPDK: 23.11 LTS
• QUIC: quiche 0.21 (Cloudflare's implementation) and msquic 2.3
• io_uring: liburing 2.6

Each test ran for 60 seconds, repeated 10 times, with the median taken. CPU frequency scaling? Disabled. Turbo boost? Disabled. IRQ affinity? Pinned to specific cores. Hyperthreading? Off. No shortcuts.

TCP: the old guard still has teeth

Plain TCP serves as the baseline everyone compares against, and its performance with modern kernels is frequently underestimated.

Standard send()/recv() over TCP with default kernel settings:

// Classic blocking TCP, nothing fancy
int sock = socket(AF_INET, SOCK_STREAM, 0);
setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, &one, sizeof(one));
connect(sock, (struct sockaddr*)&addr, sizeof(addr));

// 64KB messages, 1M iterations
for (int i = 0; i < 1000000; i++) {
 send(sock, buf, 65536, 0);
}

Results: 14.2 Gbps throughput, 11.3us mean latency, 18.7us p99.

Reasonable, but substantial performance remains untapped. The problem is that every send() call is a syscall, context switch into kernel mode, copy the buffer, context switch back.

TCP + io_uring: zero-copy sends

io_uring with IORING_OP_SEND_ZC eliminates the buffer copy:

struct io_uring ring;
io_uring_queue_init(256, &ring, 0);

struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
io_uring_prep_send_zc(sqe, sock, buf, 65536, 0, 0);
sqe->user_data = 1;
io_uring_submit(&ring);

// Wait for completion AND notification that buffer is safe to reuse
struct io_uring_cqe *cqe;
io_uring_wait_cqe(&ring, &cqe);
if (cqe->flags & IORING_CQE_F_NOTIF) {
 // Buffer can be reused now
}
io_uring_cqe_seen(&ring, cqe);

Results: 31.7 Gbps throughput, 5.8us mean latency, 9.2us p99.

That's a 2.2x throughput improvement and ~50% latency reduction. The key insight: it's not just about avoiding the copy, batching submissions into the ring means fewer syscalls overall. One io_uring_submit() can push dozens of operations.

TCP + DPDK: full kernel bypass

When you absolutely, positively need every last nanosecond:

// DPDK packet construction, you own the entire stack
struct rte_mbuf *pkt = rte_pktmbuf_alloc(mbuf_pool);
struct rte_ether_hdr *eth = rte_pktmbuf_mtod(pkt, struct rte_ether_hdr*);
// ... build ethernet, IP, TCP headers manually ...

// Direct NIC access, no kernel involvement
uint16_t nb_tx = rte_eth_tx_burst(port_id, queue_id, &pkt, 1);

Results: 82.4 Gbps throughput, 1.1us mean latency, 1.8us p99.

This is the result of removing the kernel entirely. The cost is significant: TCP must be reimplemented in userspace. In practice, this approach is reserved for specific high-frequency trading workloads, and the debugging experience requires considerable expertise.

QUIC: not just "TCP 2.0"

QUIC is frequently characterized as "the future of networking," but the reality is more nuanced. It runs in userspace, which means no kernel bypass tricks, and it mandates TLS 1.3, so every packet carries encryption overhead.

The test used Cloudflare's quiche implementation, which is extensively proven in production:

// quiche QUIC connection setup
let mut config = quiche::Config::new(quiche::PROTOCOL_VERSION)?;
config.set_application_protos(b"\x02h3")?;
config.set_max_idle_timeout(30000);
config.set_initial_max_data(10_000_000);
config.set_initial_max_stream_data_bidi_local(1_000_000);
config.enable_early_data();

let conn = quiche::connect(Some("server"), &scid, local, peer, &mut config)?;

// Send data on a stream
let written = conn.stream_send(stream_id, &buf[..65536], false)?;

Results: 8.7 Gbps throughput, 22.4us mean latency, 38.1us p99.

Slower than plain TCP? Yes. And that is on a direct connection with zero packet loss. However, QUIC was not designed to win on this benchmark. It was designed for the real internet, where packets get dropped, connections migrate between networks, and head-of-line blocking kills multiplexed HTTP/2 streams.

Where QUIC actually wins

A tc netem profile simulating a typical mobile connection was configured: 50ms RTT, 1% packet loss, 0.5% reordering.

Protocol	Throughput	p99 Latency	Connection Setup
TCP + TLS 1.3	2.1 Gbps	187ms	3 RTT (SYN + TLS)
QUIC	3.8 Gbps	112ms	1 RTT (0-RTT resume)
HTTP/3	3.6 Gbps	98ms*	0 RTT (resume)

*HTTP/3's p99 is lower because independent streams avoid HOL blocking.

On lossy networks, QUIC's 0-RTT connection resumption and independent stream multiplexing absolutely destroy TCP. The 0-RTT case is particularly brutal for TCP, by the time TCP finishes its three-way handshake and TLS negotiation, QUIC has already sent the first response.

Congestion control: the algorithm matters more than the protocol

This aspect is frequently overlooked. The choice of congestion control algorithm has more impact on real-world performance than TCP vs QUIC in most scenarios.

Four algorithms were tested on a 100ms RTT link with varying packet loss:

# Switching congestion control on Linux
sysctl -w net.ipv4.tcp_congestion_control=bbr
# or cubic, reno, bbr3 (requires patched kernel)

BBRv3 vs CUBIC vs Reno, 100ms RTT

At 0.1% loss: BBRv3 pushes 42 Gbps vs CUBIC's 28 Gbps. BBR's model-based approach doesn't back off as aggressively.

At 1% loss: BBRv3 hits 18 Gbps, CUBIC drops to 8.3 Gbps. BBR is clearly superior here.

At 5% loss: Here's the surprise, CUBIC recovers to 3.1 Gbps while BBRv3 only manages 2.4 Gbps. BBR's bandwidth probing gets confused by sustained high loss rates and oscillates between probing and draining.

Investigation into why BBRv3 underperforms at high loss rates reveals an interesting finding. The answer is in how BBR estimates the bottleneck bandwidth. It uses a windowed max filter over recent delivery rates, but at 5% loss, the ACK stream becomes so noisy that the max filter keeps picking up outlier measurements, causing the sending rate to oscillate wildly.

# Simplified BBR bandwidth estimation
class BBRBandwidthFilter:
 def __init__(self, window_size=10):
 self.window = deque(maxlen=window_size)

 def update(self, delivery_rate):
 self.window.append(delivery_rate)
 # At high loss, these samples are all over the place
 return max(self.window) # This max is the problem

HTTP/3: the full picture

HTTP/3 is QUIC with HTTP semantics on top. The performance characteristics inherit from QUIC, but there's an additional layer worth discussing: QPACK header compression.

QPACK is HTTP/3's answer to HPACK, but it's designed for QUIC's out-of-order delivery. This means it can't use the simple sequential state machine that HPACK uses. Instead, it has encoder and decoder streams that operate independently:

// QPACK dynamic table update
Encoder Stream: INSERT name=":status" value="200"
 INSERT name="content-type" value="application/json"

// Header block references dynamic table entries
Request Stream: INDEXED ref=0 (":status: 200")
 INDEXED ref=1 ("content-type: application/json")

In practice, QPACK achieves 5-15% worse compression than HPACK on typical web traffic (because it can't risk blocking), but the elimination of head-of-line blocking more than compensates.

The practical decision framework

After all these benchmarks, here is a practical recommendation framework:

Use plain TCP + io_uring when you control both endpoints, the network is reliable, and you need raw throughput. Think: inter-datacenter replication, database wire protocols, service mesh data planes.

Use QUIC when clients are on unreliable networks (mobile, WiFi), you need connection migration, or multiplexing multiple independent streams. Think: client-facing APIs, real-time communication, CDN edge.

Use DPDK only, and I mean only, when you need sub-2us latency and you have a team that can maintain a custom networking stack. Think: HFT, telco packet processing, DPI appliances.

Avoid HTTP/3 unless there is a clear requirement. If the clients are browsers, they will negotiate it automatically via Alt-Svc headers. If the clients are controlled services, the HTTP framing overhead is likely not justified, use QUIC directly or remain with TCP.

One more thing: GSO and GRO

Generic Segmentation Offload and Generic Receive Offload are the unsung heroes of modern networking performance. They let you hand the NIC a 64KB buffer and let hardware split it into MTU-sized segments:

// Enable GSO on a UDP socket (for QUIC)
int gso_size = 1200; // QUIC packet size
setsockopt(fd, SOL_UDP, UDP_SEGMENT, &gso_size, sizeof(gso_size));

// Now you can send 64KB in a single syscall
// The kernel/NIC splits it into 1200-byte UDP packets
sendmsg(fd, &msg, 0); // msg contains 64KB buffer

This single optimization can improve QUIC throughput by 3.2x. Without GSO, each QUIC packet requires a separate sendmsg() call. With GSO, you batch 50+ packets into one syscall. This is how Cloudflare gets QUIC to perform at scale.

The networking landscape is shifting fast. io_uring is getting new zero-copy features every kernel release, QUIC implementations are maturing rapidly, and kernel bypass is becoming more accessible. But the fundamentals have not changed: measure the actual workload, on the actual network, with the actual hardware. Synthetic benchmarks are a starting point, not an answer.