This blog goes from 0 to 100 really, really quickly. I have no idea what I am looking. I suppose it is not meant for beginners but it claims to be a tutorial.
This is, but only for someone who wants to do JIT work without writing assembly code, but can read assembly code back into C (or can automate that part).
Instead of doing all manual register allocations in the JIT, you get to fill in the blanks with the actual inputs after a more (maybe) diligent compiler has allocated the registers, pushed them and all that.
There's a similar set of implementation techniques in Apache Impala, where the JIT only invokes the library functions when generating JIT code, instead of writing inline JIT operations, so that they can rely on shorter compile times for the JIT and deeper optimization passes for the called functions.
My understanding is that e-graphs take care of selecting the best patch (by examining many options in parallel) but fundamentally it is still copy-and-patch.
Could you elaborate more on "fundamentally it is still copy-and-patch"? From what I can recall when I had first read about copy-and-patch a not-uncommon comparison was against Cranelift, which to me would imply that different approaches were taken. I don't recall any discussion about Cranelift's use of the technique, either, so your claim that it's at the heart of Cranelift is new information to me. Has Cranelift adopted copy-and-patch (maybe for a specific compilation stage?) in the meantime?
Interesting point about Cranelift! I've been following its development for a while, and it seems like there's always something new popping up. That connection with e-graphs adds a neat layer of complexity—it’s kinda wild to think about how optimization strategies can vary so much yet still be rooted in similar ideas.
I wonder if there's a place for copy-and-patch within Cranelift at some level, maybe for specific sequences or operations? I had a similar experience trying to streamline some code generation tasks and found that even small optimizations could lead to surprisingly big performance gains.
I think it's cool how different teams tackle the same challenges from different angles—like how CPython's JIT works, for instance. It really makes you appreciate the depth of creativity in the community. Do you think there are other JITs out there that are using these techniques in ways we haven’t seen yet? Or maybe there are trade-offs between speed and optimization that some projects have to weigh heavier than others?
Question: For what else (apart from assembler) this could be a good idea?
I think WASM, but could be for a custom byte code? and more importantly, for a set of host-native functions (like I make some rust functions that somehow exploit this idea?)
This blog goes from 0 to 100 really, really quickly. I have no idea what I am looking. I suppose it is not meant for beginners but it claims to be a tutorial.
> but it claims to be a tutorial.
This is, but only for someone who wants to do JIT work without writing assembly code, but can read assembly code back into C (or can automate that part).
Instead of doing all manual register allocations in the JIT, you get to fill in the blanks with the actual inputs after a more (maybe) diligent compiler has allocated the registers, pushed them and all that.
There's a similar set of implementation techniques in Apache Impala, where the JIT only invokes the library functions when generating JIT code, instead of writing inline JIT operations, so that they can rely on shorter compile times for the JIT and deeper optimization passes for the called functions.
Related:
Copy-and-Patch: Fast compilation for high-level languages and bytecode (2020) https://news.ycombinator.com/item?id=40553448 - June 2024 (51 comments)
A copy-and-patch JIT compiler for CPython - https://news.ycombinator.com/item?id=38769874 - Dec 2023 (68 comments)
Copy-and-Patch: Fast JIT Compilation for SQL, WebAssembly, and Others - https://news.ycombinator.com/item?id=28547057 - Sept 2021 (7 comments)
There are some experiments in using copy-and-patch for the R language (after Python): https://dl.acm.org/doi/10.1145/3759548.3763370
From a master thesis: https://www.itspy.cz/wp-content/uploads/2025/09/it_spy_2025_...
Featuring self-modifying code - it can repatch emitted instruction at runtime based on the current value type.
the accompanying post "How It Works" is worth reading alongside this tutorial
https://transactional.blog/copy-and-patch/
(key terms: abus[e|ing]: 4, force: 3, trick: 1, chance: 1)
I think this technique also lies at the heart of the Cranelift project.
https://cranelift.dev/
IIRC Cranelift doesn't use copy-and-patch. It uses e-graphs [0] as part of its optimization pipeline, though.
Closest thing in (relatively) recent news that uses copy-and-patch I can think of is CPython's new JIT.
[0]: https://github.com/bytecodealliance/rfcs/pull/27
My understanding is that e-graphs take care of selecting the best patch (by examining many options in parallel) but fundamentally it is still copy-and-patch.
Could you elaborate more on "fundamentally it is still copy-and-patch"? From what I can recall when I had first read about copy-and-patch a not-uncommon comparison was against Cranelift, which to me would imply that different approaches were taken. I don't recall any discussion about Cranelift's use of the technique, either, so your claim that it's at the heart of Cranelift is new information to me. Has Cranelift adopted copy-and-patch (maybe for a specific compilation stage?) in the meantime?
Interesting point about Cranelift! I've been following its development for a while, and it seems like there's always something new popping up. That connection with e-graphs adds a neat layer of complexity—it’s kinda wild to think about how optimization strategies can vary so much yet still be rooted in similar ideas.
I wonder if there's a place for copy-and-patch within Cranelift at some level, maybe for specific sequences or operations? I had a similar experience trying to streamline some code generation tasks and found that even small optimizations could lead to surprisingly big performance gains.
I think it's cool how different teams tackle the same challenges from different angles—like how CPython's JIT works, for instance. It really makes you appreciate the depth of creativity in the community. Do you think there are other JITs out there that are using these techniques in ways we haven’t seen yet? Or maybe there are trade-offs between speed and optimization that some projects have to weigh heavier than others?
Indeed, the original copy-and-patch paper explicitly compares against Cranelift: https://fredrikbk.com/publications/copy-and-patch.pdf
You are right. Somehow I had in my mind that e-graphs worked with pre-compiled snippets of code but it seems Cranelift does not do that.
Question: For what else (apart from assembler) this could be a good idea?
I think WASM, but could be for a custom byte code? and more importantly, for a set of host-native functions (like I make some rust functions that somehow exploit this idea?)