The Isolation Problem

Let's start with why this matters. In a serverless platform, you're running untrusted code from different tenants on shared hardware. The isolation mechanism must guarantee that:

  1. Tenant A cannot read Tenant B's memory
  2. Tenant A cannot access Tenant B's filesystem
  3. Tenant A cannot consume more CPU/memory than allocated
  4. A bug or exploit in Tenant A's code cannot compromise the host

Containers provide property 2 (via mount namespaces), property 3 (via cgroups), and partially property 1 (via PID/user namespaces). But they fail critically on property 4: all containers share the host kernel, and the Linux kernel has a massive attack surface. A kernel vulnerability means a container escape, and Linux gets kernel CVEs regularly.

Firecracker: A VMM Written in Rust

Firecracker was built by AWS specifically for Lambda and Fargate. It's a Virtual Machine Monitor (VMM) that uses KVM (Linux's built-in hypervisor) to create lightweight virtual machines. Each VM gets its own Linux kernel, its own memory space, and its own virtual devices.

The architecture:

┌─────────────────────────────────────────┐
│ HOST OS │
│ ┌──────────┐ ┌──────────┐ ┌────────┐ │
│ │Firecracker│ │Firecracker│ │ ... │ │
│ │ VMM │ │ VMM │ │ │ │
│ │ (Rust) │ │ (Rust) │ │ │ │
│ ├──────────┤ ├──────────┤ ├────────┤ │
│ │ KVM │ │ KVM │ │ KVM │ │
│ ├──────────┤ ├──────────┤ ├────────┤ │
│ │Guest │ │Guest │ │Guest │ │
│ │Linux │ │Linux │ │Linux │ │
│ │Kernel │ │Kernel │ │Kernel │ │
│ ├──────────┤ ├──────────┤ ├────────┤ │
│ │Function A│ │Function B│ │Func C │ │
│ └──────────┘ └──────────┘ └────────┘ │
└─────────────────────────────────────────┘

What makes Firecracker a "micro" VMM:

Minimal device model: Traditional VMMs (QEMU) emulate hundreds of devices, USB controllers, sound cards, legacy ISA buses. Firecracker emulates exactly 5 device types: virtio-net, virtio-block, serial console, a minimal i8042 keyboard controller (for shutdown), and a KVM clock. That's it. Fewer devices = smaller attack surface = less code to audit.

No BIOS, no UEFI: Firecracker loads the Linux kernel directly into guest memory and sets up the boot parameters manually. This skips the entire firmware boot sequence that adds seconds to traditional VM boot times.

Rate limiters built in: Every virtio device has configurable rate limiters for both bandwidth and operations per second. This is resource isolation at the VMM level, not bolted on after the fact.

Here's what Firecracker does at startup:

// Simplified Firecracker boot sequence (from the source)
fn boot_microvm(config: &VmConfig) -> Result<Vm> {
 // 1. Open /dev/kvm and create a VM
 let kvm_fd = open("/dev/kvm", O_RDWR)?;
 let vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, 0)?;

 // 2. Configure guest memory (typically 128MB-2GB)
 let guest_mem = GuestMemory::new(config.mem_size_mib)?;
 ioctl(vm_fd, KVM_SET_USER_MEMORY_REGION, &guest_mem.region())?;

 // 3. Load the kernel directly into guest memory
 let kernel = load_kernel(&config.kernel_path, &guest_mem)?;

 // 4. Set up boot parameters (command line, initrd, etc.)
 let boot_params = configure_boot_params(kernel.entry_point, &config)?;

 // 5. Create vCPUs and set registers
 let vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, 0)?;
 set_initial_registers(vcpu_fd, &boot_params)?;

 // 6. Set up virtio devices (net, block)
 let mmio_bus = setup_virtio_devices(&config, &guest_mem)?;

 // 7. Enter the VM loop (runs guest code via KVM_RUN)
 Ok(Vm { vm_fd, vcpu_fd, guest_mem, mmio_bus })
}

The VM loop itself is surprisingly simple:

loop {
 // Run guest code until it exits (I/O, interrupt, etc.)
 ioctl(vcpu_fd, KVM_RUN, 0)?;

 match vcpu_fd.exit_reason() {
 KvmExit::IoOut { port, data } => {
 // Handle I/O port access (serial console, etc.)
 handle_io_out(port, data)?;
 }
 KvmExit::MmioWrite { addr, data } => {
 // Handle MMIO access (virtio devices)
 mmio_bus.write(addr, data)?;
 }
 KvmExit::Shutdown => break,
 _ => { /* handle other exit reasons */ }
 }
}

Boot Time: Real Numbers

Firecracker boot times measured on an m5.metal instance (bare-metal, no nested virtualization):

Phase Time
─────────────────────────────────────
KVM_CREATE_VM ioctl: 0.2ms
Memory setup (256MB): 1.8ms
Kernel load (5.10, stripped): 3.1ms
vCPU setup + register init: 0.4ms
Virtio device setup: 0.8ms
─────────────────────────────────────
VMM ready to KVM_RUN: 6.3ms
Guest kernel boot to init: ~125ms
Alpine Linux init to shell: ~30ms
─────────────────────────────────────
Total to running userspace: ~160ms

160ms from nothing to a running Linux userspace. That's remarkable, but for serverless functions that need to respond in <200ms, that boot time is still a significant chunk of the latency budget. This is where snapshot-restore comes in.

Firecracker Snapshots

Firecracker can snapshot a running VM's entire state (CPU registers, memory, device state) to disk, and restore it in ~5ms:

# Create a snapshot of a running VM
curl --unix-socket /tmp/firecracker.socket -X PUT \
 http://localhost/snapshot/create \
 -d '{
 "snapshot_type": "Full",
 "snapshot_path": "/snapshots/function-v1.snap",
 "mem_file_path": "/snapshots/function-v1.mem"
 }'

# Restore a VM from snapshot (~5ms)
curl --unix-socket /tmp/firecracker.socket -X PUT \
 http://localhost/snapshot/load \
 -d '{
 "snapshot_path": "/snapshots/function-v1.snap",
 "mem_file_path": "/snapshots/function-v1.mem"
 }'

AWS Lambda uses this exact mechanism: when you deploy a function, Lambda boots it once, snapshots the initialized state, and restores from snapshot on subsequent cold starts. This is why Lambda cold starts are ~200ms for most runtimes instead of the 5+ seconds you'd expect from a full VM boot.

Unikraft: Library Operating Systems

Unikernels take a fundamentally different approach. Instead of running a full Linux kernel with thousands of syscalls, drivers, and subsystems, a unikernel includes only the OS components that the application actually uses.

Unikraft is the most actively developed unikernel framework. It's a library OS, you pick the components you need:

Traditional VM:
 Application → libc → Linux Kernel (25M+ lines of code) → Hardware

Unikraft Unikernel:
 Application → lib/nolibc → lib/vfscore → lib/9pfs → KVM driver
 → lib/uknetdev → virtio-net driver
 → lib/uksched → cooperative scheduler
 (total: ~100K lines of code)

A Unikraft unikernel is compiled into a single binary that boots directly on KVM (or Xen):

# Unikraft application configuration (kraft.yaml)
spec: v0.6

unikraft:
 version: stable
 kconfig:
 CONFIG_LIBUKBOOT: y
 CONFIG_LIBUKLIBPARAM: y
 CONFIG_LIBPOSIX_PROCESS: y

libraries:
 musl:
 version: stable
 lwip:
 version: stable
 kconfig:
 CONFIG_LWIP_TCP: y
 CONFIG_LWIP_DHCP: y

targets:
 - platform: qemu
 architecture: x86_64
 - platform: firecracker
 architecture: x86_64

Build and run:

# Build the unikernel
kraft build --target firecracker/x86_64 --log-level debug

# Run with Firecracker
kraft run --target firecracker/x86_64 \
 --memory 64 \
 --network bridge:kraft0 \
 -- /my-http-server --port 8080

Head-to-Head Comparison

All measurements on the same hardware (c5.metal, bare-metal):

 Firecracker Unikraft Docker
 (Alpine guest) (HTTP server) (Alpine)
────────────────────────────────────────────────────────────────────────────
Boot to first HTTP response: 160ms 8ms ~800ms*
Memory footprint (idle): 32MB 2MB ~15MB
Image size: ~50MB ~1.2MB ~8MB
Supported syscalls: ~400 ~160 ~400
Security boundary: KVM + VMM KVM Namespace
Debugging experience: Good (gdb) Painful Excellent
Ecosystem compatibility: Linux binaries Recompile req'd Everything
────────────────────────────────────────────────────────────────────────────
* Docker measured including containerd/shim overhead

That 8ms boot time for Unikraft is not a typo. A unikernel has no BIOS, no bootloader, no kernel module loading, no systemd, no init process. It goes directly from "VM started" to "application main() running." The memory footprint tells the same story, 2MB versus 32MB because there's no kernel memory management overhead, no process scheduler, no driver framework for devices that don't exist.

But look at the ecosystem compatibility row. Firecracker runs any Linux binary, Node.js, Python, Go, Java, anything with a Linux build. Unikraft requires recompilation against its libc (musl-based) and may not support all syscalls your application uses. In one case, two days were spent trying to get a Python Flask app running on Unikraft before discovering that it used sendfile() which was not implemented in Unikraft's network stack at the time.

The Security Model Comparison

Both approaches use KVM for memory isolation, so the hardware-enforced boundary is identical. The difference is in the attack surface within the VM:

Firecracker MicroVM attack surface:
 - Guest Linux kernel (~25M lines of C)
 - Firecracker VMM (~50K lines of Rust)
 - 5 virtual device implementations
 Total guest attack surface: Large kernel, but isolated by KVM

Unikraft unikernel attack surface:
 - Unikraft kernel (~100K lines of C)
 - Only included libraries
 - Same 5 virtual device implementations (Firecracker backend)
 Total guest attack surface: Minimal kernel, isolated by KVM

If an attacker achieves code execution within the guest, a Unikraft unikernel gives them almost nothing to work with, no shell, no filesystem utilities, no kernel exploit gadgets. A Firecracker guest running Alpine Linux has the full Linux kernel attack surface from within.

However, in practice, the KVM boundary is what matters most. The guest kernel attack surface is relevant only if the attacker has already compromised the application, at which point you're in damage-limitation mode regardless.

Practical Production Choices

For serverless function isolation: Firecracker. The ecosystem compatibility is worth the extra memory and boot time. With snapshot-restore, cold starts are ~5ms anyway. The operational maturity is unmatched, AWS runs billions of Lambda invocations per day on Firecracker.

For edge computing with known, controlled workloads: Unikraft. When the application's behavior is well-characterized (serve static files, run WASM modules, proxy HTTP), a 2MB unikernel that boots in 8ms is irresistible. The operational overhead is higher because debugging is harder, but for well-characterized workloads, it's worth it.

For everything else: Firecracker with Kata Containers. This gives you container ergonomics (docker build, docker push, standard CI/CD) with VM isolation. The boot time overhead is ~150ms, which is acceptable for most workloads that aren't latency-critical serverless functions.

The honest truth is that unikernels have been "the future" for a decade, and they're still not mainstream. Firecracker succeeded where unikernels haven't because it made a pragmatic tradeoff: accept a full Linux kernel with its overhead and attack surface, but make it fast enough and isolated enough that it doesn't matter. Sometimes good enough, shipped and battle-tested, beats perfect and still-maturing.