What's the difference between a container runtime and a container engine?

The container engine (Docker, Podman) handles image management, networking, and orchestration. The container runtime (runc, youki, kata) is the low-level component that actually creates and runs the container process using Linux kernel features like namespaces and cgroups.

Is youki production-ready?

Yes. youki passes the full OCI runtime-spec compliance test suite and is used in production by several organizations. It's a drop-in replacement for runc with better startup performance and memory safety guarantees from Rust.

When should I use kata-containers instead of runc?

Use kata-containers when you need hardware-level isolation for untrusted workloads, multi-tenant platforms, running user-submitted code, or compliance requirements that mandate VM isolation. The trade-off is ~150ms additional startup time and higher memory overhead per container.

Container Runtime Internals: runc, youki, kata-containers, and Sandbox Isolation Models

What actually happens when you start a container

When you run docker run -it ubuntu bash, Docker calls containerd, which calls a "shim" process, which calls the OCI runtime (runc by default). The OCI runtime is where the actual container creation happens. Here's the sequence:

docker run → dockerd → containerd → containerd-shim → runc create → runc start
 │
 ├── clone(CLONE_NEWNS | CLONE_NEWPID | ...)
 ├── mount rootfs
 ├── pivot_root
 ├── setup cgroups
 ├── apply seccomp filter
 ├── drop capabilities
 └── execve("/bin/bash")

runc itself is a ~15,000 line Go program. It does five fundamental things:

Creates Linux namespaces to isolate the container's view of the system
Sets up a filesystem (mount namespace + pivot_root)
Configures cgroups to limit resource usage
Applies security policies (seccomp, capabilities, AppArmor/SELinux)
Executes the container process

Let's dig into each one.

Namespaces: the isolation foundation

Linux namespaces partition kernel resources so that one set of processes sees one set of resources, and another set sees a different set. Containers typically use 7 namespace types:

// This is essentially what runc does with clone()
#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/wait.h>

int child_fn(void *arg) {
 // Inside the container, we have our own PID namespace
 printf("Container PID: %d\n", getpid()); // Always 1
 printf("Container hostname: ");
 sethostname("container", 9);

 char hostname[64];
 gethostname(hostname, sizeof(hostname));
 printf("%s\n", hostname);

 // Mount proc for the new PID namespace
 mount("proc", "/proc", "proc", 0, NULL);

 execve("/bin/sh", (char*[]){"/bin/sh", NULL}, NULL);
 return 0;
}

int main() {
 char stack[1024 * 1024];

 int flags = CLONE_NEWNS // Mount namespace
 | CLONE_NEWPID // PID namespace
 | CLONE_NEWUTS // Hostname namespace
 | CLONE_NEWNET // Network namespace
 | CLONE_NEWIPC // IPC namespace
 | CLONE_NEWUSER // User namespace
 | CLONE_NEWCGROUP; // Cgroup namespace

 pid_t pid = clone(child_fn, stack + sizeof(stack), flags | SIGCHLD, NULL);
 waitpid(pid, NULL, 0);
 return 0;
}

The user namespace is the most interesting one for security. It maps the container's UID 0 (root) to an unprivileged UID on the host:

# Check user namespace mapping
$ cat /proc/<container-pid>/uid_map
 0 100000 65536
# Container UID 0 → Host UID 100000
# Container UID 1 → Host UID 100001
# ... up to 65536 UIDs

This means even if a process escapes the container as "root," it's actually UID 100000 on the host, an unprivileged user. This is rootless containers in a nutshell.

cgroup v2: resource control done right

cgroup v1 was a mess. Each resource controller (cpu, memory, io, pids) had its own hierarchy, and a process could be in different cgroups for different controllers. cgroup v2 unifies everything into a single hierarchy:

# cgroup v2 unified hierarchy
/sys/fs/cgroup/
├── system.slice/
│ ├── docker-abc123.scope/
│ │ ├── cgroup.controllers # "cpu memory io pids"
│ │ ├── cpu.max # "100000 100000" (100% of 1 CPU)
│ │ ├── memory.max # "536870912" (512MB)
│ │ ├── memory.swap.max # "0" (no swap)
│ │ ├── io.max # "8:0 rbps=104857600 wbps=52428800"
│ │ ├── pids.max # "1024"
│ │ └── cgroup.procs # PIDs in this cgroup
│ └── docker-def456.scope/
│ └── ...

The killer feature of cgroup v2 is the pressure stall information (PSI):

# How much time are tasks in this cgroup stalled on resources?
$ cat /sys/fs/cgroup/system.slice/docker-abc123.scope/cpu.pressure
some avg10=4.52 avg60=2.18 avg300=1.05 total=182394
full avg10=0.00 avg60=0.00 avg300=0.00 total=0

$ cat /sys/fs/cgroup/system.slice/docker-abc123.scope/memory.pressure
some avg10=0.00 avg60=0.00 avg300=0.00 total=0
full avg10=0.00 avg60=0.00 avg300=0.00 total=0

PSI tells you how much a cgroup is affected by resource contention, not just whether it hit a limit. "some avg10=4.52" means that in the last 10 seconds, at least some tasks were stalled waiting for CPU 4.52% of the time. This is dramatically more useful than the boolean "did we OOM" signal from v1.

Here's how runc sets up cgroups programmatically:

// Simplified from runc's cgroup setup
func setupCgroupV2(cgroupPath string, resources *specs.LinuxResources) error {
 if err := os.MkdirAll(cgroupPath, 0755); err != nil {
 return err
 }

 // CPU: 50% of one core (50000us out of 100000us period)
 if resources.CPU != nil {
 quota := fmt.Sprintf("%d %d", *resources.CPU.Quota, *resources.CPU.Period)
 os.WriteFile(filepath.Join(cgroupPath, "cpu.max"), []byte(quota), 0644)
 }

 // Memory: hard limit
 if resources.Memory != nil && resources.Memory.Limit != nil {
 limit := strconv.FormatInt(*resources.Memory.Limit, 10)
 os.WriteFile(filepath.Join(cgroupPath, "memory.max"), []byte(limit), 0644)
 // Disable swap
 os.WriteFile(filepath.Join(cgroupPath, "memory.swap.max"), []byte("0"), 0644)
 }

 // PID limit, prevents fork bombs
 if resources.Pids != nil {
 limit := strconv.FormatInt(resources.Pids.Limit, 10)
 os.WriteFile(filepath.Join(cgroupPath, "pids.max"), []byte(limit), 0644)
 }

 // Move the container's init process into this cgroup
 return os.WriteFile(
 filepath.Join(cgroupPath, "cgroup.procs"),
 []byte(strconv.Itoa(containerPid)),
 0644,
 )
}

It's just writing to files. That's cgroups, the kernel exposes a filesystem interface, and you write configuration values to files. Beautifully simple, terrifyingly powerful.

youki: runc in Rust

youki is a rewrite of runc in Rust, and it's not just a vanity project. There are real engineering reasons to care:

Startup time: compare runtimes with the same image, kernel, cgroup configuration and repeated runs. The youki repository is the implementation reference. This guide does not include a reproducible startup benchmark.

Memory safety: Rust can prevent categories of memory errors in safe code. It does not automatically prevent file descriptor leaks or container escapes. CVE 2024 21626 concerns leaked file descriptors and requires examining runtime behavior, not inferring safety from the implementation language.

Drop-in replacement: youki passes the full OCI runtime-spec compliance tests. You can literally swap it in:

# Replace runc with youki in containerd
# /etc/containerd/config.toml
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
 runtime_type = "io.containerd.runc.v2"
 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
 BinaryName = "/usr/local/bin/youki"

Restart containerd, and every container on the machine now uses youki. I did this on a staging cluster three months ago and literally nobody noticed. That's the best kind of infrastructure change.

kata-containers: when namespaces aren't enough

A fundamental limitation of standard containers is that namespaces are a userspace isolation mechanism. They partition the kernel's view of resources, but every container shares the same kernel. A kernel exploit in one container compromises every container on the host.

kata-containers solve this by running each container inside a lightweight VM (microVM). The container sees a dedicated kernel:

Standard container:
┌──────────┐ ┌──────────┐ ┌──────────┐
│Container1│ │Container2│ │Container3│
├──────────┴─┴──────────┴─┴──────────┤
│ Host Kernel │ ← Shared attack surface
├─────────────────────────────────────┤
│ Hardware │
└─────────────────────────────────────┘

kata-container:
┌──────────┐ ┌──────────┐ ┌──────────┐
│Container1│ │Container2│ │Container3│
├──────────┤ ├──────────┤ ├──────────┤
│ Guest K │ │ Guest K │ │ Guest K │ ← Separate kernels
├──────────┤ ├──────────┤ ├──────────┤
│ VMM │ │ VMM │ │ VMM │ ← QEMU/Cloud-hypervisor/Firecracker
├──────────┴─┴──────────┴─┴──────────┤
│ Host Kernel │
└─────────────────────────────────────┘

The performance overhead is real but manageable:

# Container startup time comparison
$ time crictl run --runtime=runc pod.json container.json
# real 0.087s

$ time crictl run --runtime=kata pod.json container.json
# real 0.241s (~150ms overhead for VM boot)

# Memory overhead per container
# runc: ~5MB (just the container process + shim)
# kata: ~40-80MB (guest kernel + minimal userspace + VMM)

kata supports three VMMs:

QEMU: Full-featured, supports most hardware. Heavier (~130MB memory overhead).
Cloud Hypervisor: Rust-based, minimal VMM designed for containers. ~40MB overhead.
Firecracker: Amazon's microVM monitor. ~30MB overhead, fastest boot.

For our multi-tenant platform, we use kata with Cloud Hypervisor for customer workloads and runc for our own infrastructure services. The RuntimeClass API in Kubernetes makes this trivial:

# RuntimeClass for kata
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
 name: kata-clh
handler: kata-clh
overhead:
 podFixed:
 memory: "60Mi"
 cpu: "50m"
scheduling:
 nodeSelector:
 kata-runtime: "true"

---
# Pod using kata
apiVersion: v1
kind: Pod
metadata:
 name: untrusted-workload
spec:
 runtimeClassName: kata-clh
 containers:
 - name: user-code
 image: customer-function:latest
 resources:
 limits:
 memory: "256Mi"
 cpu: "500m"

gVisor: the middle ground

gVisor (runsc) takes a different approach. Instead of running a full VM, it interposes a userspace kernel that intercepts syscalls from the container process:

gVisor architecture:
┌──────────────────┐
│ Container App │
├──────────────────┤
│ Sentry │ ← Userspace kernel (intercepts syscalls)
│ (implements │
│ ~380 syscalls) │
├──────────────────┤
│ Gofer │ ← Filesystem proxy (sandboxed)
├──────────────────┤
│ Host Kernel │ ← Only Sentry talks to the host kernel
└──────────────────┘

The Sentry reimplements Linux syscalls in Go. Your container process thinks it's talking to a Linux kernel, but it's actually talking to gVisor. If a syscall triggers a kernel vulnerability, the exploit only compromises the Sentry process, not the host kernel.

# Install and configure gVisor
$ gvisor install
$ cat /etc/containerd/config.toml
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
 runtime_type = "io.containerd.runsc.v1"

# Run with gVisor
$ docker run --runtime=runsc -it ubuntu bash

# Check syscalls available inside the container
$ strace -c ls 2>&1 | head -20
# Only ~240 of the 380+ Linux syscalls are implemented

The trade-offs compared to kata:

Startup: gVisor adds ~50ms (vs kata's ~150ms)
Memory: gVisor uses ~15MB overhead (vs kata's ~40-80MB)
Compatibility: gVisor doesn't support all syscalls, applications that use uncommon syscalls (like io_uring, specific NUMA operations) may not work
Performance: Syscall-heavy workloads take a 2-10x performance hit because every syscall goes through the Sentry

seccomp: the last line of defense

Regardless of which runtime you use, seccomp (secure computing mode) filters the syscalls a container can make. The default Docker seccomp profile blocks 44 syscalls out of ~380+, including dangerous ones like kexec_load (load a new kernel), reboot, and mount.

{
 "defaultAction": "SCMP_ACT_ERRNO",
 "defaultErrnoRet": 1,
 "syscalls": [
 {
 "names": ["accept", "bind", "clone", "close", "connect",
 "execve", "exit", "fcntl", "fstat", "futex",
 "getpid", "ioctl", "mmap", "mprotect", "nanosleep",
 "open", "openat", "poll", "read", "recvfrom",
 "rt_sigaction", "sendto", "socket", "write"],
 "action": "SCMP_ACT_ALLOW"
 },
 {
 "names": ["clone3"],
 "action": "SCMP_ACT_ALLOW",
 "args": [
 {
 "index": 0,
 "value": 2114060288,
 "op": "SCMP_CMP_MASKED_EQ"
 }
 ]
 }
 ]
}

Here's a practical gotcha: the default profile doesn't allow io_uring syscalls (io_uring_setup, io_uring_enter, io_uring_register). If you're running a modern application that uses io_uring for async I/O, it will silently fall back to slower syscalls or fail entirely. I've debugged this exact issue twice.

# Profile your actual application to find needed syscalls
$ strace -c -f your-application 2>&1 | tail -20
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
 35.42 0.182346 12 15234 write
 22.18 0.114218 8 14028 read
 15.73 0.081002 27 3001 futex
 8.91 0.045889 45 1020 io_uring_enter # Need this!
 ...

The runtime decision tree

After running all four runtimes in production, here's how I decide:

runc: Default for trusted, internal workloads. Low overhead, maximum compatibility, well-understood security model. 90% of containers should run on runc.

youki: Drop-in runc replacement when you want better startup time and Rust's memory safety guarantees. No reason not to use it if you're comfortable with newer software.

kata-containers: Untrusted workloads that need VM-level isolation. Multi-tenant platforms, FaaS, running customer code. Accept the ~150ms startup overhead and ~40MB memory overhead.

gVisor: When you need better isolation than runc but can't afford kata's resource overhead. Good for batch processing of untrusted data, web servers handling untrusted input. Watch out for syscall compatibility.

The container ecosystem is converging on a layered security model: namespaces for basic isolation, cgroups for resource limits, seccomp for syscall filtering, and optional VM/sandbox layers for defense in depth. Understanding these layers, and knowing which knobs to turn, is the difference between containers that are "probably fine" and containers you'd bet your company on.

Container Runtime Internals: runc, youki, kata-containers, and Sandbox Isolation Models