{"schemaVersion":1,"id":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design","title":"Async Rust Internals: Tokio Runtime Design, Work-Stealing, and Building a Minimal Runtime from Scratch","description":"A deep technical exploration of how Tokio actually works, the work-stealing scheduler, I/O driver, timer wheel, waker mechanism, and the Pin story. Then we build a minimal async runtime from scratch to solidify the mental model.","language":"en","datePublished":"2025-11-15","dateModified":null,"contentFormat":"text/markdown","content":"Rust\n\n# Async Rust Internals: Tokio Runtime Design, Work-Stealing, and Building a Minimal Runtime from Scratch\n\n2025-11-158 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/async-rust-tokio-internals-runtime-design>)\n\n## The Async Mental Model\n\nIn Rust, an ` async fn ` compiles down to a state machine that implements the ` Future ` trait:\n\n```rust\ntrait Future {\n type Output;\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;\n}\n```\n\nEvery ` .await ` point becomes a state transition. The compiler generates something roughly like this for a simple async function:\n\n```rust\nasync fn fetch_and_parse(url: &str) -> Result<Data> {\n let response = reqwest::get(url).await?; // state 0 → state 1\n let text = response.text().await?; // state 1 → state 2\n serde_json::from_str(&text).map_err(Into::into)\n}\n\n// Conceptually compiles to:\nenum FetchAndParse {\n State0 { url: String },\n State1 { response: Response },\n State2 { text: String },\n}\n\nimpl Future for FetchAndParse {\n type Output = Result<Data>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {\n loop {\n match self.state {\n State0 { ref url } => {\n match self.get_future.poll(cx) {\n Poll::Ready(resp) => self.transition_to(State1 { response: resp? }),\n Poll::Pending => return Poll::Pending,\n }\n }\n State1 { ref response } => {\n match self.text_future.poll(cx) {\n Poll::Ready(text) => self.transition_to(State2 { text: text? }),\n Poll::Pending => return Poll::Pending,\n }\n }\n State2 { ref text } => {\n return Poll::Ready(serde_json::from_str(text).map_err(Into::into));\n }\n }\n }\n }\n}\n```\n\nThe critical insight: when ` poll ` returns ` Poll::Pending `, the future goes to sleep. Nothing is checking on it. The ` Waker ` stored in ` Context ` is the mechanism by which the I/O system says \"hey, this socket has data now, poll that future again.\"\n\n## Tokio's Architecture\n\n[Tokio](<https://tokio.rs/>) has four major subsystems:\n\n```\n┌──────────────────────────────────────────┐\n│ Tokio Runtime │\n├──────────┬──────────┬──────────┬─────────┤\n│ Scheduler│ I/O │ Timer │ Blocking│\n│ (work- │ Driver │ Driver │ Thread │\n│ stealing)│ (epoll/ │ (timer │ Pool │\n│ │ kqueue/ │ wheel) │ │\n│ │ io_uring)│ │ │\n└──────────┴──────────┴──────────┴─────────┘\n```\n\n### The Work-Stealing Scheduler\n\nTokio runs N worker threads (default: number of CPU cores). Each worker has a local run queue (a fixed-size ring buffer of 256 tasks) plus access to a global injection queue.\n\nWhen a worker finishes polling a task:\n\n1. Check its local queue for the next task\n2. If empty, check the global injection queue\n3. If empty, steal from another worker's local queue (take half their tasks)\n4. If everything's empty, park the thread\n\n```rust\n// Simplified work-stealing loop\nfn worker_loop(worker: &Worker, shared: &Shared) {\n loop {\n // 1. Local queue\n if let Some(task) = worker.local_queue.pop() {\n task.poll();\n continue;\n }\n\n // 2. Global queue\n if let Some(task) = shared.global_queue.pop() {\n // Also grab a batch to amortize lock overhead\n while let Some(extra) = shared.global_queue.try_pop() {\n worker.local_queue.push(extra);\n }\n task.poll();\n continue;\n }\n\n // 3. Steal from a random other worker\n let victim = random_worker(shared.workers.len());\n if let Some(stolen) = shared.workers[victim].local_queue.steal_half() {\n stolen.first().poll();\n for task in stolen.rest() {\n worker.local_queue.push(task);\n }\n continue;\n }\n\n // 4. Nothing to do, park\n worker.park();\n }\n}\n```\n\nThe steal-half strategy is important. If worker A has 100 tasks queued and worker B is idle, B steals 50 tasks at once. This amortizes the synchronization cost and quickly balances the load.\n\nI profiled this with ` tokio-console ` on a production service handling 50K req/s. The steal rate stabilized at about 3% of task executions, meaning workers were well-balanced and rarely needed to steal. When I introduced a CPU-heavy task on one endpoint, the steal rate jumped to 18%, and latency on *other* endpoints increased. That's when I moved the CPU work to ` spawn_blocking `.\n\n### The I/O Driver\n\nTokio's I/O driver wraps the OS event notification system, ` epoll ` on Linux, ` kqueue ` on macOS, ` io_uring ` on newer Linux kernels (behind a feature flag). The driver runs on a dedicated thread (or shares time with worker threads, depending on configuration).\n\n```rust\n// Simplified I/O driver loop\nfn io_driver_loop(driver: &mut IoDriver) {\n let mut events = Events::with_capacity(1024);\n\n loop {\n // Block until I/O events arrive or timeout\n driver.poll.poll(&mut events, Some(Duration::from_millis(61)))?;\n\n for event in &events {\n let token = event.token();\n if let Some(waker) = driver.registry.get_waker(token) {\n // Wake the future that was waiting for this I/O\n waker.wake();\n }\n }\n }\n}\n```\n\nWhen you call ` TcpStream::read `, Tokio:\n\n1. Attempts a non-blocking read\n2. If ` WouldBlock `, registers interest with epoll via the I/O driver\n3. Returns ` Poll::Pending ` with the waker stored in the registry\n4. When data arrives, epoll fires, the driver wakes the task\n5. The scheduler re-polls the future, and this time the read succeeds\n\n### The Timer Wheel\n\nTokio's timer system uses a hierarchical timer wheel, the same data structure used in Linux kernel timers. It's a multi-level array where each level covers a different time granularity:\n\n```\nLevel 0: 64 slots × 1ms = 64ms range\nLevel 1: 64 slots × 64ms = ~4s range\nLevel 2: 64 slots × 4s = ~4min range\nLevel 3: 64 slots × 4min = ~4.5hr range\nLevel 4: 64 slots × 4.5hr = ~12 day range\nLevel 5: 64 slots × 12d = ~2 year range\n```\n\nInserting and cancelling a timer is O(1). Advancing the wheel to fire expired timers is amortized O(1) per timer fired. This is dramatically better than a binary heap (O(log n)) when you have thousands of active timers, which is common in connection timeout management.\n\n```rust\n// How tokio::time::sleep works under the hood\npub async fn sleep(duration: Duration) {\n let deadline = Instant::now() + duration;\n let entry = timer_wheel.insert(deadline);\n\n poll_fn(|cx| {\n if Instant::now() >= deadline {\n Poll::Ready(())\n } else {\n entry.set_waker(cx.waker().clone());\n Poll::Pending\n }\n }).await;\n}\n```\n\n### The Waker Mechanism\n\nThe ` Waker ` is the glue that connects I/O readiness to task scheduling. When a future returns ` Poll::Pending `, it stores the waker. When the event happens, the waker is called, which pushes the task back onto the scheduler's queue.\n\n```rust\n// Tokio's waker implementation (simplified)\nimpl Wake for TaskHarness {\n fn wake(self: Arc<Self>) {\n // Set the NOTIFIED flag\n self.state.fetch_or(NOTIFIED, Ordering::AcqRel);\n\n // Push task to the scheduler\n self.scheduler.schedule(self.clone());\n }\n}\n```\n\nThe ` Arc<TaskHarness> ` is critical, it means the waker keeps the task alive and can be cloned cheaply. Multiple I/O sources can hold wakers for the same task (think ` select! `), and whichever fires first wakes the task.\n\n## Building a Minimal Runtime\n\nLet's build the simplest possible async runtime to nail down the concepts. No work-stealing, no I/O driver, just a single-threaded executor that can run futures to completion.\n\n```rust\nuse std::collections::VecDeque;\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::sync::{Arc, Mutex};\nuse std::task::{Context, Poll, Wake, Waker};\n\ntype BoxFuture = Pin<Box<dyn Future<Output = ()> + Send>>;\n\nstruct MiniRuntime {\n queue: Arc<Mutex<VecDeque<Arc<Task>>>>,\n}\n\nstruct Task {\n future: Mutex<Option<BoxFuture>>,\n queue: Arc<Mutex<VecDeque<Arc<Task>>>>,\n}\n\nimpl Wake for Task {\n fn wake(self: Arc<Self>) {\n // Re-enqueue the task when woken\n self.queue.lock().unwrap().push_back(self.clone());\n }\n}\n\nimpl MiniRuntime {\n fn new() -> Self {\n MiniRuntime {\n queue: Arc::new(Mutex::new(VecDeque::new())),\n }\n }\n\n fn spawn(&self, future: impl Future<Output = ()> + Send + 'static) {\n let task = Arc::new(Task {\n future: Mutex::new(Some(Box::pin(future))),\n queue: self.queue.clone(),\n });\n self.queue.lock().unwrap().push_back(task);\n }\n\n fn run(&self) {\n loop {\n let task = {\n let mut queue = self.queue.lock().unwrap();\n match queue.pop_front() {\n Some(task) => task,\n None => break, // No more tasks\n }\n };\n\n let waker = Waker::from(task.clone());\n let mut cx = Context::from_waker(&waker);\n\n let mut future_slot = task.future.lock().unwrap();\n if let Some(mut future) = future_slot.take() {\n match future.as_mut().poll(&mut cx) {\n Poll::Ready(()) => {} // Task complete, drop it\n Poll::Pending => {\n // Put the future back, it'll be re-enqueued when woken\n *future_slot = Some(future);\n }\n }\n }\n }\n }\n}\n```\n\nThis is about 50 lines and it actually works for futures that wake themselves synchronously. To make it useful for real I/O, you'd need to add an epoll loop that calls the appropriate wakers when file descriptors become ready, essentially the I/O driver.\n\n```rust\nfn main() {\n let rt = MiniRuntime::new();\n\n rt.spawn(async {\n println!(\"Task 1: start\");\n // yield_now would re-enqueue via waker\n println!(\"Task 1: end\");\n });\n\n rt.spawn(async {\n println!(\"Task 2: start\");\n println!(\"Task 2: end\");\n });\n\n rt.run();\n}\n```\n\n## Real-World Gotchas\n\nA few things I've hit in production that stem from not understanding these internals:\n\n**Holding a MutexGuard across .await:** This blocks the worker thread, and since Tokio is cooperative, no other task on that worker can run until the lock is released. Use ` tokio::sync::Mutex ` if you must hold a lock across await points.\n\n```rust\n// BAD, blocks the worker thread\nlet data = std_mutex.lock().unwrap();\nsome_async_operation(&data).await;\ndrop(data);\n\n// GOOD, async-aware mutex\nlet data = tokio_mutex.lock().await;\nsome_async_operation(&data).await;\ndrop(data);\n```\n\n**Too many tasks with short polls:** Each task has overhead, the ` Arc<TaskHarness> `, the waker allocation, the scheduling. If you spawn 100,000 tasks that each do one tiny operation, the scheduling overhead dominates. Batch them.\n\n**Forgetting ` spawn_blocking ` for file I/O:** ` std::fs ` operations are synchronous and will block the worker thread. Tokio's ` tokio::fs ` is just a wrapper around ` spawn_blocking ` + ` std::fs `. Use it, or use ` spawn_blocking ` directly.\n\n```rust\n// This blocks a worker thread, don't do this in async context\nlet data = std::fs::read_to_string(\"config.json\")?;\n\n// This runs on the blocking thread pool\nlet data = tokio::fs::read_to_string(\"config.json\").await?;\n```\n\n## Summary\n\nTokio's design is a masterclass in systems programming trade-offs. The work-stealing scheduler gives you CPU efficiency. The I/O driver gives you scalability. The timer wheel gives you performance with thousands of concurrent timeouts. And the entire thing is built on Rust's ownership model, which means data races are caught at compile time rather than at 3 AM in production.\n\nUnderstanding these internals won't change how you write basic async code. But it will change how you debug it, how you structure your task hierarchy, and how you avoid the performance cliffs that catch everyone eventually.\n\n## Frequently asked questions\n\n### Why does Rust need an external runtime for async instead of building one into the language?\n\nRust's design philosophy separates the language primitives (Future trait, async/await syntax, Pin) from the executor that drives them. This means you can use Tokio for network servers, smol for lightweight tasks, or Embassy for embedded systems, all with the same async/await syntax. Baking a runtime into the language would force compromises that work for nobody.\n\n### How does Tokio's work-stealing scheduler differ from OS thread scheduling?\n\nOS schedulers handle preemptive multitasking, they can interrupt threads at any point. Tokio's scheduler is cooperative, tasks must yield at .await points. Work-stealing means idle worker threads can grab tasks from busy workers' queues, improving CPU utilization. The combination gives you M:N threading, many tasks on fewer OS threads, without preemption overhead.\n\n### When should I use tokio::spawn vs tokio::spawn\\_blocking?\n\nUse tokio::spawn for async work that will .await on I/O or other futures. Use spawn\\_blocking for synchronous code that would block the thread, CPU-intensive computation, synchronous file I/O, or calling C libraries. spawn\\_blocking runs on a separate thread pool so it won't starve the async workers.\n\n## Tips\n\n1. **LuchoTip:** Never call ` .await ` inside a ` spawn_blocking ` closure, and never call blocking code inside an async task without ` spawn_blocking `. Mixing these two worlds is the #1 cause of mysterious Tokio deadlocks I've debugged.\n2. **LuchoTip:** If your Tokio app's latency spikes under load, check ` tokio_unstable ` + ` tokio-console ` first. Nine times out of ten it's a task holding a ` MutexGuard ` across an ` .await ` point, starving the work-stealing scheduler.\n3. **LuchoTip:** Set ` worker_threads ` to your CPU count, not higher. More threads than cores means context-switching overhead eats the concurrency gains. I benchmarked a 16-core machine with 16 vs 32 workers, 32 was 11% *slower*.\n\n- rust\n- async\n- tokio\n- runtime\n- work-stealing\n- futures\n- waker\n- io-driver\n- concurrency\n- systems-programming\n\n[Back to Blog](<https://lucioduran.com/blog>)","headings":[{"level":1,"text":"Async Rust Internals: Tokio Runtime Design, Work-Stealing, and Building a Minimal Runtime from Scratch","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#title"},{"level":2,"text":"The Async Mental Model","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-async-mental-model"},{"level":2,"text":"Tokio's Architecture","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#tokios-architecture"},{"level":3,"text":"The Work-Stealing Scheduler","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-work-stealing-scheduler"},{"level":3,"text":"The I/O Driver","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-io-driver"},{"level":3,"text":"The Timer Wheel","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-timer-wheel"},{"level":3,"text":"The Waker Mechanism","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-waker-mechanism"},{"level":2,"text":"Building a Minimal Runtime","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#building-a-minimal-runtime"},{"level":2,"text":"Real-World Gotchas","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#real-world-gotchas"},{"level":2,"text":"Summary","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#summary"},{"level":2,"text":"Frequently asked questions","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#questions"},{"level":3,"text":"Why does Rust need an external runtime for async instead of building one into the language?","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#question-1"},{"level":3,"text":"How does Tokio's work-stealing scheduler differ from OS thread scheduling?","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#question-2"},{"level":3,"text":"When should I use tokio::spawn vs tokio::spawn_blocking?","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#question-3"},{"level":2,"text":"Tips","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#article-tips"}],"sections":[{"heading":"Async Rust Internals: Tokio Runtime Design, Work-Stealing, and Building a Minimal Runtime from Scratch","level":1,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design","content":"Rust"},{"heading":"Async Rust Internals: Tokio Runtime Design, Work-Stealing, and Building a Minimal Runtime from Scratch","level":1,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#title","content":"2025-11-158 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/async-rust-tokio-internals-runtime-design>)"},{"heading":"The Async Mental Model","level":2,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-async-mental-model","content":"In Rust, an ` async fn ` compiles down to a state machine that implements the ` Future ` trait:\n\n```rust\ntrait Future {\n type Output;\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;\n}\n```\n\nEvery ` .await ` point becomes a state transition. The compiler generates something roughly like this for a simple async function:\n\n```rust\nasync fn fetch_and_parse(url: &str) -> Result<Data> {\n let response = reqwest::get(url).await?; // state 0 → state 1\n let text = response.text().await?; // state 1 → state 2\n serde_json::from_str(&text).map_err(Into::into)\n}\n\n// Conceptually compiles to:\nenum FetchAndParse {\n State0 { url: String },\n State1 { response: Response },\n State2 { text: String },\n}\n\nimpl Future for FetchAndParse {\n type Output = Result<Data>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {\n loop {\n match self.state {\n State0 { ref url } => {\n match self.get_future.poll(cx) {\n Poll::Ready(resp) => self.transition_to(State1 { response: resp? }),\n Poll::Pending => return Poll::Pending,\n }\n }\n State1 { ref response } => {\n match self.text_future.poll(cx) {\n Poll::Ready(text) => self.transition_to(State2 { text: text? }),\n Poll::Pending => return Poll::Pending,\n }\n }\n State2 { ref text } => {\n return Poll::Ready(serde_json::from_str(text).map_err(Into::into));\n }\n }\n }\n }\n}\n```\n\nThe critical insight: when ` poll ` returns ` Poll::Pending `, the future goes to sleep. Nothing is checking on it. The ` Waker ` stored in ` Context ` is the mechanism by which the I/O system says \"hey, this socket has data now, poll that future again.\""},{"heading":"Tokio's Architecture","level":2,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#tokios-architecture","content":"[Tokio](<https://tokio.rs/>) has four major subsystems:\n\n```\n┌──────────────────────────────────────────┐\n│ Tokio Runtime │\n├──────────┬──────────┬──────────┬─────────┤\n│ Scheduler│ I/O │ Timer │ Blocking│\n│ (work- │ Driver │ Driver │ Thread │\n│ stealing)│ (epoll/ │ (timer │ Pool │\n│ │ kqueue/ │ wheel) │ │\n│ │ io_uring)│ │ │\n└──────────┴──────────┴──────────┴─────────┘\n```"},{"heading":"The Work-Stealing Scheduler","level":3,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-work-stealing-scheduler","content":"Tokio runs N worker threads (default: number of CPU cores). Each worker has a local run queue (a fixed-size ring buffer of 256 tasks) plus access to a global injection queue.\n\nWhen a worker finishes polling a task:\n\n1. Check its local queue for the next task\n2. If empty, check the global injection queue\n3. If empty, steal from another worker's local queue (take half their tasks)\n4. If everything's empty, park the thread\n\n```rust\n// Simplified work-stealing loop\nfn worker_loop(worker: &Worker, shared: &Shared) {\n loop {\n // 1. Local queue\n if let Some(task) = worker.local_queue.pop() {\n task.poll();\n continue;\n }\n\n // 2. Global queue\n if let Some(task) = shared.global_queue.pop() {\n // Also grab a batch to amortize lock overhead\n while let Some(extra) = shared.global_queue.try_pop() {\n worker.local_queue.push(extra);\n }\n task.poll();\n continue;\n }\n\n // 3. Steal from a random other worker\n let victim = random_worker(shared.workers.len());\n if let Some(stolen) = shared.workers[victim].local_queue.steal_half() {\n stolen.first().poll();\n for task in stolen.rest() {\n worker.local_queue.push(task);\n }\n continue;\n }\n\n // 4. Nothing to do, park\n worker.park();\n }\n}\n```\n\nThe steal-half strategy is important. If worker A has 100 tasks queued and worker B is idle, B steals 50 tasks at once. This amortizes the synchronization cost and quickly balances the load.\n\nI profiled this with ` tokio-console ` on a production service handling 50K req/s. The steal rate stabilized at about 3% of task executions, meaning workers were well-balanced and rarely needed to steal. When I introduced a CPU-heavy task on one endpoint, the steal rate jumped to 18%, and latency on *other* endpoints increased. That's when I moved the CPU work to ` spawn_blocking `."},{"heading":"The I/O Driver","level":3,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-io-driver","content":"Tokio's I/O driver wraps the OS event notification system, ` epoll ` on Linux, ` kqueue ` on macOS, ` io_uring ` on newer Linux kernels (behind a feature flag). The driver runs on a dedicated thread (or shares time with worker threads, depending on configuration).\n\n```rust\n// Simplified I/O driver loop\nfn io_driver_loop(driver: &mut IoDriver) {\n let mut events = Events::with_capacity(1024);\n\n loop {\n // Block until I/O events arrive or timeout\n driver.poll.poll(&mut events, Some(Duration::from_millis(61)))?;\n\n for event in &events {\n let token = event.token();\n if let Some(waker) = driver.registry.get_waker(token) {\n // Wake the future that was waiting for this I/O\n waker.wake();\n }\n }\n }\n}\n```\n\nWhen you call ` TcpStream::read `, Tokio:\n\n1. Attempts a non-blocking read\n2. If ` WouldBlock `, registers interest with epoll via the I/O driver\n3. Returns ` Poll::Pending ` with the waker stored in the registry\n4. When data arrives, epoll fires, the driver wakes the task\n5. The scheduler re-polls the future, and this time the read succeeds"},{"heading":"The Timer Wheel","level":3,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-timer-wheel","content":"Tokio's timer system uses a hierarchical timer wheel, the same data structure used in Linux kernel timers. It's a multi-level array where each level covers a different time granularity:\n\n```\nLevel 0: 64 slots × 1ms = 64ms range\nLevel 1: 64 slots × 64ms = ~4s range\nLevel 2: 64 slots × 4s = ~4min range\nLevel 3: 64 slots × 4min = ~4.5hr range\nLevel 4: 64 slots × 4.5hr = ~12 day range\nLevel 5: 64 slots × 12d = ~2 year range\n```\n\nInserting and cancelling a timer is O(1). Advancing the wheel to fire expired timers is amortized O(1) per timer fired. This is dramatically better than a binary heap (O(log n)) when you have thousands of active timers, which is common in connection timeout management.\n\n```rust\n// How tokio::time::sleep works under the hood\npub async fn sleep(duration: Duration) {\n let deadline = Instant::now() + duration;\n let entry = timer_wheel.insert(deadline);\n\n poll_fn(|cx| {\n if Instant::now() >= deadline {\n Poll::Ready(())\n } else {\n entry.set_waker(cx.waker().clone());\n Poll::Pending\n }\n }).await;\n}\n```"},{"heading":"The Waker Mechanism","level":3,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#the-waker-mechanism","content":"The ` Waker ` is the glue that connects I/O readiness to task scheduling. When a future returns ` Poll::Pending `, it stores the waker. When the event happens, the waker is called, which pushes the task back onto the scheduler's queue.\n\n```rust\n// Tokio's waker implementation (simplified)\nimpl Wake for TaskHarness {\n fn wake(self: Arc<Self>) {\n // Set the NOTIFIED flag\n self.state.fetch_or(NOTIFIED, Ordering::AcqRel);\n\n // Push task to the scheduler\n self.scheduler.schedule(self.clone());\n }\n}\n```\n\nThe ` Arc<TaskHarness> ` is critical, it means the waker keeps the task alive and can be cloned cheaply. Multiple I/O sources can hold wakers for the same task (think ` select! `), and whichever fires first wakes the task."},{"heading":"Building a Minimal Runtime","level":2,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#building-a-minimal-runtime","content":"Let's build the simplest possible async runtime to nail down the concepts. No work-stealing, no I/O driver, just a single-threaded executor that can run futures to completion.\n\n```rust\nuse std::collections::VecDeque;\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::sync::{Arc, Mutex};\nuse std::task::{Context, Poll, Wake, Waker};\n\ntype BoxFuture = Pin<Box<dyn Future<Output = ()> + Send>>;\n\nstruct MiniRuntime {\n queue: Arc<Mutex<VecDeque<Arc<Task>>>>,\n}\n\nstruct Task {\n future: Mutex<Option<BoxFuture>>,\n queue: Arc<Mutex<VecDeque<Arc<Task>>>>,\n}\n\nimpl Wake for Task {\n fn wake(self: Arc<Self>) {\n // Re-enqueue the task when woken\n self.queue.lock().unwrap().push_back(self.clone());\n }\n}\n\nimpl MiniRuntime {\n fn new() -> Self {\n MiniRuntime {\n queue: Arc::new(Mutex::new(VecDeque::new())),\n }\n }\n\n fn spawn(&self, future: impl Future<Output = ()> + Send + 'static) {\n let task = Arc::new(Task {\n future: Mutex::new(Some(Box::pin(future))),\n queue: self.queue.clone(),\n });\n self.queue.lock().unwrap().push_back(task);\n }\n\n fn run(&self) {\n loop {\n let task = {\n let mut queue = self.queue.lock().unwrap();\n match queue.pop_front() {\n Some(task) => task,\n None => break, // No more tasks\n }\n };\n\n let waker = Waker::from(task.clone());\n let mut cx = Context::from_waker(&waker);\n\n let mut future_slot = task.future.lock().unwrap();\n if let Some(mut future) = future_slot.take() {\n match future.as_mut().poll(&mut cx) {\n Poll::Ready(()) => {} // Task complete, drop it\n Poll::Pending => {\n // Put the future back, it'll be re-enqueued when woken\n *future_slot = Some(future);\n }\n }\n }\n }\n }\n}\n```\n\nThis is about 50 lines and it actually works for futures that wake themselves synchronously. To make it useful for real I/O, you'd need to add an epoll loop that calls the appropriate wakers when file descriptors become ready, essentially the I/O driver.\n\n```rust\nfn main() {\n let rt = MiniRuntime::new();\n\n rt.spawn(async {\n println!(\"Task 1: start\");\n // yield_now would re-enqueue via waker\n println!(\"Task 1: end\");\n });\n\n rt.spawn(async {\n println!(\"Task 2: start\");\n println!(\"Task 2: end\");\n });\n\n rt.run();\n}\n```"},{"heading":"Real-World Gotchas","level":2,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#real-world-gotchas","content":"A few things I've hit in production that stem from not understanding these internals:\n\n**Holding a MutexGuard across .await:** This blocks the worker thread, and since Tokio is cooperative, no other task on that worker can run until the lock is released. Use ` tokio::sync::Mutex ` if you must hold a lock across await points.\n\n```rust\n// BAD, blocks the worker thread\nlet data = std_mutex.lock().unwrap();\nsome_async_operation(&data).await;\ndrop(data);\n\n// GOOD, async-aware mutex\nlet data = tokio_mutex.lock().await;\nsome_async_operation(&data).await;\ndrop(data);\n```\n\n**Too many tasks with short polls:** Each task has overhead, the ` Arc<TaskHarness> `, the waker allocation, the scheduling. If you spawn 100,000 tasks that each do one tiny operation, the scheduling overhead dominates. Batch them.\n\n**Forgetting ` spawn_blocking ` for file I/O:** ` std::fs ` operations are synchronous and will block the worker thread. Tokio's ` tokio::fs ` is just a wrapper around ` spawn_blocking ` + ` std::fs `. Use it, or use ` spawn_blocking ` directly.\n\n```rust\n// This blocks a worker thread, don't do this in async context\nlet data = std::fs::read_to_string(\"config.json\")?;\n\n// This runs on the blocking thread pool\nlet data = tokio::fs::read_to_string(\"config.json\").await?;\n```"},{"heading":"Summary","level":2,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#summary","content":"Tokio's design is a masterclass in systems programming trade-offs. The work-stealing scheduler gives you CPU efficiency. The I/O driver gives you scalability. The timer wheel gives you performance with thousands of concurrent timeouts. And the entire thing is built on Rust's ownership model, which means data races are caught at compile time rather than at 3 AM in production.\n\nUnderstanding these internals won't change how you write basic async code. But it will change how you debug it, how you structure your task hierarchy, and how you avoid the performance cliffs that catch everyone eventually."},{"heading":"Frequently asked questions","level":2,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#questions","content":""},{"heading":"Why does Rust need an external runtime for async instead of building one into the language?","level":3,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#question-1","content":"Rust's design philosophy separates the language primitives (Future trait, async/await syntax, Pin) from the executor that drives them. This means you can use Tokio for network servers, smol for lightweight tasks, or Embassy for embedded systems, all with the same async/await syntax. Baking a runtime into the language would force compromises that work for nobody."},{"heading":"How does Tokio's work-stealing scheduler differ from OS thread scheduling?","level":3,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#question-2","content":"OS schedulers handle preemptive multitasking, they can interrupt threads at any point. Tokio's scheduler is cooperative, tasks must yield at .await points. Work-stealing means idle worker threads can grab tasks from busy workers' queues, improving CPU utilization. The combination gives you M:N threading, many tasks on fewer OS threads, without preemption overhead."},{"heading":"When should I use tokio::spawn vs tokio::spawn_blocking?","level":3,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#question-3","content":"Use tokio::spawn for async work that will .await on I/O or other futures. Use spawn\\_blocking for synchronous code that would block the thread, CPU-intensive computation, synchronous file I/O, or calling C libraries. spawn\\_blocking runs on a separate thread pool so it won't starve the async workers."},{"heading":"Tips","level":2,"url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#article-tips","content":"1. **LuchoTip:** Never call ` .await ` inside a ` spawn_blocking ` closure, and never call blocking code inside an async task without ` spawn_blocking `. Mixing these two worlds is the #1 cause of mysterious Tokio deadlocks I've debugged.\n2. **LuchoTip:** If your Tokio app's latency spikes under load, check ` tokio_unstable ` + ` tokio-console ` first. Nine times out of ten it's a task holding a ` MutexGuard ` across an ` .await ` point, starving the work-stealing scheduler.\n3. **LuchoTip:** Set ` worker_threads ` to your CPU count, not higher. More threads than cores means context-switching overhead eats the concurrency gains. I benchmarked a 16-core machine with 16 vs 32 workers, 32 was 11% *slower*.\n\n- rust\n- async\n- tokio\n- runtime\n- work-stealing\n- futures\n- waker\n- io-driver\n- concurrency\n- systems-programming\n\n[Back to Blog](<https://lucioduran.com/blog>)"}],"links":[{"text":"Español","url":"https://lucioduran.com/es/blog/async-rust-tokio-internals-runtime-design"},{"text":"Tokio","url":"https://tokio.rs/"},{"text":"Back to Blog","url":"https://lucioduran.com/blog"}],"structuredData":[{"@context":"https://schema.org","@graph":[{"@type":"BlogPosting","@id":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design#article","mainEntityOfPage":{"@type":"WebPage","@id":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design"},"headline":"Async Rust Internals: Tokio Runtime Design, Work-Stealing, and Building a Minimal Runtime from Scratch","description":"A deep technical exploration of how Tokio actually works, the work-stealing scheduler, I/O driver, timer wheel, waker mechanism, and the Pin story. Then we build a minimal async runtime from scratch to solidify the mental model.","datePublished":"2025-11-15","author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"publisher":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"image":"https://lucioduran.com/favicons/og-default.png","url":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design","inLanguage":"en","isPartOf":{"@id":"https://lucioduran.com/blog#blog"},"articleSection":"Rust","keywords":"rust, async, tokio, runtime, work-stealing, futures, waker, io-driver, concurrency, systems-programming","wordCount":1520,"timeRequired":"PT8M","workTranslation":[{"@type":"BlogPosting","inLanguage":"es","url":"https://lucioduran.com/es/blog/async-rust-tokio-internals-runtime-design"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://lucioduran.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://lucioduran.com/blog"},{"@type":"ListItem","position":3,"name":"Async Rust Internals: Tokio Runtime Design, Work-Stealing, and Building a Minimal Runtime from Scratch","item":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design"}]},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"Why does Rust need an external runtime for async instead of building one into the language?","acceptedAnswer":{"@type":"Answer","text":"Rust's design philosophy separates the language primitives (Future trait, async/await syntax, Pin) from the executor that drives them. This means you can use Tokio for network servers, smol for lightweight tasks, or Embassy for embedded systems, all with the same async/await syntax. Baking a runtime into the language would force compromises that work for nobody."}},{"@type":"Question","name":"How does Tokio's work-stealing scheduler differ from OS thread scheduling?","acceptedAnswer":{"@type":"Answer","text":"OS schedulers handle preemptive multitasking, they can interrupt threads at any point. Tokio's scheduler is cooperative, tasks must yield at .await points. Work-stealing means idle worker threads can grab tasks from busy workers' queues, improving CPU utilization. The combination gives you M:N threading, many tasks on fewer OS threads, without preemption overhead."}},{"@type":"Question","name":"When should I use tokio::spawn vs tokio::spawn_blocking?","acceptedAnswer":{"@type":"Answer","text":"Use tokio::spawn for async work that will .await on I/O or other futures. Use spawn_blocking for synchronous code that would block the thread, CPU-intensive computation, synchronous file I/O, or calling C libraries. spawn_blocking runs on a separate thread pool so it won't starve the async workers."}}]}]}],"contentHash":"746281c6e0b78f32f0c221eeb95e296ff6ca822715e1b153054a0f02176b3429","provenance":{"method":"Deterministic serialization of the server-rendered main element","source":"https://lucioduran.com/blog/async-rust-tokio-internals-runtime-design","hashAlgorithm":"sha256","hashScope":"JSON document fields before contentHash and provenance","sourceHTMLBytes":66309,"contentBytes":13602}}