Building a Wasm Runner with Cloud Hypervisor

2025-12-16T00:00:00+00:00

In this post, I will demonstrate how to run WebAssembly (Wasm) inside a MicroVM. I'll provide context first, then focus on a few selected implementation details.

Wasm for Extensibility</h2>
Architecture defines what parts of a system are stable and what parts are dynamic. System extensions are a prime example of dynamic components, but they raise significant security and observability concerns—especially when the code is untrusted.
Consider this scenario:

I need to run user-provided code.</li>
I do not trust this code.</li>
The system must load, execute, and potentially unload the code at runtime without build dependencies.</li>
It must be efficient.</li>
It must support user-friendly debugging.</li> </ul>
This mirrors the challenges faced by cloud providers and security-conscious platform engineers.
WebAssembly (Wasm) is an excellent fit for this because:

Many languages compile to Wasm.</li>
Wasm is designed for a specific execution environment (the runtime).</li>
The runtime is extensible.</li> </ul>
The Wasm runtime allows running user code that interacts with the "outside" through a well-defined interface. Note that "runtime" here is synonymous with "execution environment" or "virtual machine" (like the JVM).
In summary, Wasm allows us to run code while the runtime manages the API and isolation. Problem solved? Not quite.
Sandboxing the Sandbox</h2>
The isolation provided by a Wasm runtime reduces the privileges of user code (sandboxing), but risks remain:

The Wasm runtime itself may have vulnerabilities.</li>
The API exposed to user code might inadvertently allow unsafe access to the operating system.</li> </ul>
Wasm acts as a security boundary, but mistakes are possible. Two levels of isolation provide significantly better security.
In this post, we will set up a "MicroVM" to execute Wasm payloads. We will build a minimal Function-as-a-Service (FaaS) runtime from scratch using Rust, Cloud Hypervisor</a>, and Wasmtime. This setup will boot a Linux kernel, load a WebAssembly module over a virtual socket, and execute it.
If you are familiar with Microsoft's Hyperlight-Wasm</a>, this approach is similar but focuses on the fundamentals.
To follow along, you need a Linux machine with KVM enabled and the ability to configure and build a Linux kernel. If you are using a cloud VPS, ensure nested virtualization is enabled.
Unlike many minimal tutorials that use static linking (musl), we will use a dynamically linked C runtime. This allows us to use the standard system unwinder (glibc), providing full panic stack traces inside the VM—crucial for debugging failing user code.
The Architecture</h3>
Cloud Hypervisor is a virtual machine monitor (VMM) written in Rust, descending from the CrosVM and Firecracker family. It simulates hardware abstractions rather than executing high-level language instructions.
We will simulate a machine that boots a minimal Linux kernel, which then executes the Wasm runtime as its only process.
We will use "Direct Kernel Boot" with the following components:

The Hypervisor: Cloud Hypervisor (built in Rust, runs on top of KVM).</li>
The Guest Kernel: A minimal Linux kernel (`vmlinux</code>).</li>`
`The Guest OS: A custom Initramfs containing our Rust binary and necessary shared libraries (libc.so</code>, libgcc</code>, etc.).</li>`
`The Payload: A Wasm module that performs pure computation.</li>`
`The Transport: VirtIO-VSOCK, providing a direct pipe between host and guest.</li> </ul>`Step 1: The Minimal Kernel</h3> Cloud Hypervisor works best with a kernel tuned for PVH booting (skipping legacy BIOS/UEFI). Follow instructions on Cloud Hypervisor docs</a> or build the kernel your own way using make menuconfig</code>. Ensure the following are set to Yes (not Module): CONFIG_PVH=y</code>: For fast booting.</li> CONFIG_VIRTIO_PCI=y</code>: To see virtual hardware.</li> CONFIG_VIRTIO_VSOCKETS=y</code>: For communication.</li> CONFIG_BLK_DEV_INITRD=y</code>: To load the custom binary.</li> </ul> Step 2: The Wasm Payload</h3> We need code to run. We’ll compile this Rust function to wasm32-unknown-unknown</code> to keep it "pure"—no system calls, just math and memory. wasm_payload/src/lib.rs // A 1MB static buffer inside the Wasm linear memory const BUF_SIZE: usize = 1024 * 1024; static mut BUFFER: [u8; BUF_SIZE] = [0; BUF_SIZE]; #[unsafe(no_mangle)] #[allow(static_mut_refs)] pub extern "C" fn get_buffer_ptr() -> mut u8 { unsafe { BUFFER.as_mut_ptr() } } #[unsafe(no_mangle)] pub extern "C" fn process_data(len: usize) -> usize { let slice = unsafe { &mut BUFFER[0..len] }; // Example: Reverse the bytes slice.reverse(); // Return the length processed len } </code></pre> Compile it: cargo build --target wasm32-unknown-unknown --release </code></pre> If you have not compiled to Wasm before, you may want to run rustup target add wasm32-unknown-unknown</code>. Step 3: The Guest "Init" (The Micro-OS)</h3> We will write a Rust program that acts as the entire guest operating system (init process). It boots, listens on a socket, runs Wasm, and shuts down. Interaction happens via system calls, mediated by the C runtime library. We use a Dual-Socket design: Port 1000 (Control): Metadata, Wasm binary uploads, status signals.</li> Port 1001 (Data): Raw data stream.</li> </ul> This allows streaming data directly from the socket into the Wasm memory buffer. Zero-copy isn't possible here because we must copy data into the Wasm module instance's memory to maintain isolation. The following demo code implements this, using tokio (1, "full" features), tokio-vsock (0.3) and Wasmtime (24.0) API calls. guest_init/src/main.rs use tokio::io::{AsyncReadExt, AsyncWriteExt}; use tokio_vsock::VsockListener; use wasmtime::; const CMD_PORT: u32 = 1000; const DATA_PORT: u32 = 1001; #[tokio::main] async fn main() { println!("[Guest] Booting Dual-Socket Runner..."); // 1. Setup Wasm Engine let mut config = Config::new(); config.async_support(true); let engine = Engine::new(&config).unwrap(); let linker = Linker::new(&engine); let mut store = Store::new(&engine, ()); // 2. Start Listeners let mut cmd_listener = VsockListener::bind(u32::MAX, CMD_PORT).expect("Bind CMD failed"); let mut data_listener = VsockListener::bind(u32::MAX, DATA_PORT).expect("Bind DATA failed"); println!("[Guest] Waiting for Host to connect..."); // 3. Accept Connections (Sequential for simplicity) // Host must connect to CMD first, then DATA. let (mut cmd_stream, _) = cmd_listener.accept().await.unwrap(); println!("[Guest] CMD connected."); let (mut data_stream, _) = data_listener.accept().await.unwrap(); println!("[Guest] DATA connected."); // 4. Receive Wasm Binary (over CMD channel for reliability) let mut size_buf = [0u8; 4]; cmd_stream.read_exact(&mut size_buf).await.unwrap(); let wasm_len = u32::from_be_bytes(size_buf) as usize; println!("[Guest] Downloading Wasm module ({} bytes)...", wasm_len); let mut wasm_binary = vec![0u8; wasm_len]; cmd_stream.read_exact(&mut wasm_binary).await.unwrap(); // 5. Instantiate let module = Module::new(&engine, &wasm_binary).unwrap(); let instance = linker.instantiate_async(&mut store, &module).await.unwrap(); let memory = instance.get_memory(&mut store, "memory") .expect("No memory exported"); let get_ptr = instance.get_typed_func::<(), i32>(&mut store, "get_buffer_ptr") .unwrap(); let process_func = instance.get_typed_func::<i32, i32>(&mut store, "process_data") .unwrap(); let wasm_ptr = get_ptr.call_async(&mut store, ()).await.unwrap() as usize; println!("[Guest] Wasm Ready. Entering Processing Loop."); let _ = cmd_stream.write_all(b"READY").await; // Signal Host // 6. Processing Loop loop { // A. Wait for Command: "Process X bytes" let mut job_header = [0u8; 4]; if cmd_stream.read_exact(&mut job_header).await.is_err() { break; } let data_len = u32::from_be_bytes(job_header) as usize; eprintln!("Expecting Payload len {data_len}!"); // B. READ DIRECTLY TO WASM MEMORY // We get a mutable slice of Wasm memory. let mem_data = memory.data_mut(&mut store); // Safety check bounds if wasm_ptr + data_len > mem_data.len() { eprintln!("Payload too big for Wasm buffer!"); break; } let wasm_slice = &mut mem_data[wasm_ptr .. wasm_ptr + data_len]; // This is the key step: Kernel copies socket buffer -> Wasm Memory directly. data_stream.read_exact(wasm_slice).await.unwrap(); eprintln!("Read data!!"); // C. EXECUTE process_func.call_async(&mut store, data_len as i32).await.unwrap(); eprintln!("Made call!!"); // D. WRITE DIRECTLY FROM WASM MEMORY let mem_data = memory.data(&store); // Re-borrow immutable let result_slice = &mem_data[wasm_ptr .. wasm_ptr + data_len]; eprintln!("Writing result!!"); data_stream.write_all(result_slice).await.unwrap(); eprintln!("Writing done!!"); // E. Ack let _ = cmd_stream.write_all(b"DONE").await; } } </code></pre> Building the Image</h3> We gather all files for the guest filesystem. Apart from the guest init binary, we need some files to support dynamic linking of the C runtime library. Many tutorials suggest compiling with x86_64-unknown-linux-musl</code> to get a single static binary. While this is small (5MB), it breaks the ability to get stack traces because the musl unwinder often conflicts with JIT compilers like Wasmtime. Instead, we compile normally which on Linux will link dynamically against glibc. We then bundle the shared libraries and ship them into the VM. cargo build --release </code></pre> We use a script to automatically find the required .so</code> files (libc.so.6</code>, libgcc_s.so.1</code>, etc.) and package them into the initramfs. Please note that my cargo outputs binaries into /tmp</code>, you may need to adjust the paths to the binary. package_vm.sh #!/bin/bash set -e APP_NAME="guest_init" BIN_PATH="/tmp/target/release/$APP_NAME" BUILD_DIR="./build_output" # 1. Clean and Create Staging Area rm -rf $BUILD_DIR mkdir -p $BUILD_DIR/lib64 mkdir -p $BUILD_DIR/lib/x86_64-linux-gnu # 2. Copy the Main Binary to /init echo "Copying binary..." cp $BIN_PATH $BUILD_DIR/init # 3. Find and Copy Shared Libraries echo "Resolving dependencies..." # Get list of dependencies using ldd # Output format is usually: "libname.so => /path/to/libname.so (address)" DEPENDENCIES=$(ldd $BIN_PATH | awk '{if ($3 != "") print $3; else print $1}') for DEP in $DEPENDENCIES; do # Skip virtual dynamically linked things like linux-vdso.so.1 if [[ "$DEP" == "vdso" ]]; then continue fi # Check if file exists if [ -f "$DEP" ]; then echo " Bundling: $DEP" # We need to copy it to the SAME path structure relative to root # e.g., /lib/x86_64-linux-gnu/libc.so.6 # --> build_output/lib/x86_64-linux-gnu/libc.so.6 DIRname=$(dirname "$DEP") mkdir -p "$BUILD_DIR$DIRname" cp "$DEP" "$BUILD_DIR$DEP" else echo " Warning: Could not find $DEP" fi done # 4. Handle the Dynamic Linker explicitly # ldd output often shows the loader as a full path, but we need to ensure # the interpreter path hardcoded in the binary exists. INTERPRETER=$(readelf -l $BIN_PATH \ | grep "interpreter" | awk -F': ' '{print $2}' | tr -d ']') echo " Bundling Interpreter: $INTERPRETER" if [ -f "$INTERPRETER" ]; then DIRname=$(dirname "$INTERPRETER") mkdir -p "$BUILD_DIR$DIRname" cp "$INTERPRETER" "$BUILD_DIR$INTERPRETER" fi # 5. Create CPIO echo "Creating initramfs.cpio..." cd $BUILD_DIR find . -print0 | cpio --null -ov --format=newc > ../initramfs.cpio cd .. echo "Done! Dynamic initramfs ready." </code></pre> This increases the image size to ~25MB, but in return, if our Rust code panics or Wasmtime crashes, we get a full, readable stack trace in the console. At this point, we have neatly packaged up everything we need into a CPIO archive that we will use with Cloud Hypervisor's direct kernel boot method in a bit. Step 4: The Host Driver</h3> We implement the host driver in Rust to orchestrate the VM. It connects to the Cloud Hypervisor via the Unix domain socket /tmp/ch_vsock.sock</code>. The "Connect String" Mechanism Cloud Hypervisor uses a Unix domain socket on the host to expose the VSOCK capability. It acts as a bridge. However, unlike a direct socket connection where you just connect and talk, Cloud Hypervisor acts as a multiplexer. When we connect to the Unix socket, we are talking to the Hypervisor, not the guest yet. We must tell the Hypervisor which guest port we want to reach. We do this by sending a text command: CONNECT <PORT>\n</code> If the connection is successful, Cloud Hypervisor will respond with: OK <PORT>\n</code> Only after receiving this acknowledgement can we start sending raw binary data intended for the guest. If we skip this, our data will be interpreted as commands by the Hypervisor, causing errors. host_runner/src/main.rs use tokio::io::{AsyncReadExt, AsyncWriteExt}; use tokio::net::UnixStream; use std::time::Duration; const SOCKET_PATH: &str = "/tmp/ch_vsock.sock"; const CMD_PORT: u32 = 1000; const DATA_PORT: u32 = 1001; #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { println!("[Host] Connecting to Guest via {}...", SOCKET_PATH); // 1. Connect to Guest (Retry loop until Guest is up) let mut cmd_stream = connect_via_unix(SOCKET_PATH, CMD_PORT).await?; let mut data_stream = connect_via_unix(SOCKET_PATH, DATA_PORT).await?; // 2. Read Wasm Binary let wasm_path = "target/wasm32-unknown-unknown/release/payload.wasm"; let wasm_binary = std::fs::read(wasm_path).expect("Failed to read Wasm file"); let wasm_len = wasm_binary.len() as u32; // 3. Send Wasm Binary println!("[Host] Sending Wasm binary..."); cmd_stream.write_all(&wasm_len.to_be_bytes()).await?; cmd_stream.write_all(&wasm_binary).await?; // 4. Wait for Ready Signal from Guest let mut buf = [0u8; 5]; // "READY" cmd_stream.read_exact(&mut buf).await?; assert_eq!(&buf, b"READY"); println!("[Host] Guest is READY."); // 5. Send Data Job let payload = b"Hello MicroVM!"; let payload_len = payload.len() as u32; cmd_stream.write_all(&payload_len.to_be_bytes()).await?; data_stream.write_all(payload).await?; // 6. Receive Result let mut result_buf = vec![0u8; payload_len as usize]; data_stream.read_exact(&mut result_buf).await?; println!("[Host] Result: {:?}", String::from_utf8_lossy(&result_buf)); Ok(()) } async fn connect_via_unix(path: &str, port: u32) -> Result<UnixStream, Box<dyn std::error::Error>> { loop { match UnixStream::connect(path).await { Ok(mut stream) => { // Handshake for Cloud Hypervisor let msg = format!("CONNECT {}\n", port); stream.write_all(msg.as_bytes()).await?; // Read Response: "OK <port>\n" let mut resp = String::new(); loop { let mut b = [0u8; 1]; stream.read_exact(&mut b).await?; let c = b[0] as char; resp.push(c); if c == '\n' { break; } } if !resp.starts_with("OK") { return Err(format!("Handshake failed: {}", resp.trim()).into()); } return Ok(stream); }, Err(_) => { // Wait for Cloud Hypervisor to create the socket tokio::time::sleep(Duration::from_millis(500)).await; } } } } </code></pre> Step 5: Launch!</h3> We invoke Cloud Hypervisor, pointing it to our kernel, our CPIO (initramfs), and enabling the VSOCK device. sudo ./cloud-hypervisor \ --kernel ./vmlinux \ --cmdline "console=hvc0 quiet" \ --initramfs ./initramfs.cpio \ --vsock cid=3,socket=/tmp/ch_vsock.sock </code></pre> In a separate terminal, run the Host Runner: cargo run --release --bin host_runner </code></pre> Summary</h3> We just built a secure, isolated runtime where: Isolation: The code runs in a VM, not a container. The kernel enforces the boundary.</li> Observability: By shipping glibc, we have full debugging capabilities inside the guest.</li> Speed: By stripping the OS to the bare essentials and using VSOCK, overhead is minimal.</li> </ul> There is a lot more here that can be done, like using virtiofs to expose selected files on the host to the guest, tuning the kernel to optimize boot time, or system programming interface, all the way to implementing hypercalls. This architecture is the foundation of modern high-density serverless platforms. You now have the blueprint to build your own.

Indexed Reverse Polish Notation, an Alternative to AST

2025-12-12T00:00:00+00:00

"Why study compiler construction? Because knowing how a programming language is specified and implemented makes you a better programmer." I still remember these words, pronounced as matter-of-fact introduction to an undergrad course of compilers.
Compiler engineers have come up with many useful programming techniques and representations.
Today, I want to write about one such technique, an alternative to Abstract Syntax Trees (ASTs). Inspired by the parse tree representation in the Carbon compiler</a>, this post explains a way to represent parsed source code using a variation of Reverse Polish Notation (RPN), in a contiguous array.
We call this Indexed RPN. Ordering program parts in a linear sequence very naturally leads to machine interpretation, which is well-known for calculators but maybe a little less well-known when there are scoped definitions and control flow structures.
This is by no means a new way of doing things, but with modern machines having plenty of memory, there may have been less pressure to reach for techniques that memory-friendly.
1. From Arithmetic to Indices</h2>
Let’s start with an arithmetic expression example: We want to represent (3 + 4) * 5.
In a standard AST, this is a tree of pointers. In a standard stack-machine RPN, this looks like 3 4 + 5 *. Now we want something slightly different, because we want to operate on tree structure. For example, if we deal with this expression in a compiler that translates and optimizes. We want to be able to refer to specific sub-expressions later.
The "Administrative Normal Form" Perspective</h3>
Before we look at the memory layout, let's imagine breaking up this expression by naming subexpression. If we had local definitions in our language this would give us Administrative Normal Form (ANF). In ANF, we give a name to every intermediate result:
// Source: (3 + 4) * 5 val t0 = 3 val t1 = 4 val t2 = add(t0, t1) val t3 = 5 val t4 = mul(t2, t3) </code></pre> The expression now takes a lot more to write, but for a compiler it is much more structured. The arguments of the operations are always names, which makes the data flow and also the order in which arguments get evaluated fully explicit. Here, t2</code> depends entirely on t0</code> and t1</code>, and t0</code> is evaluated before t1</code>. The In-Memory Representation</h3> We don't want local definitions just yet, the above is just to motivate flattening of a tree structure. If we store the instructions in a contiguous array (e.g., a vector or Vec), the index of the node becomes its name. The internal names are merely indices into the sequence of nodes. Index (ID)</th> Node Kind</th> Operands / Data</th></tr></thead> 0</td> IntLiteral</code></td> 3</code></td></tr> 1</td> IntLiteral</code></td> 4</code></td></tr> 2</td> BinaryOp</code></td> Add(lhs: 0, rhs: 1)</code></td></tr> 3</td> IntLiteral</code></td> 5</code></td></tr> 4</td> BinaryOp</code></td> Mul(lhs: 2, rhs: 3)</code></td></tr> </tbody></table> This is similar to Reverse Polish Notation (RPN), but there is a difference. In standard RPN, there is an implicit stack from which Add</code> consumes items blindly. In Indexed RPN, Add</code> explicitly refers to indices 0 and 1. This provides a stable reference to every sub-expression, allowing us to traverse the code and locate nodes without necessarily having to build up a stack. 2. Dealing with "Let" and Scope</h2> Let us make the language more realistic by adding local let-declarations and scoping. // Source let a = 10; let b = a + 5; </code></pre> Here we face a problem: Source variables (a, b) are different from our internal indices (0, 1, 2...). We need a node that bridges this gap — an "Introducer". The Bind Node</h3> We introduce a Bind node. This node represents the action of bringing a name into existence in the current scope. Depending on the language you are working with, a binding may have semantic significance. For example, if references to the binding are objects of the language like Rust or C++ references. Index</th> Node Kind</th> Operands</th> Meaning</th></tr></thead> 0</td> IntLiteral</code></td> 10</code></td> The raw value 10.</td></tr> 1</td> Bind</code></td> name: "a", val: 0</code></td> Introducer: "a" exists, bound to Index 0.</td></tr> 2</td> NameRef</code></td> ref: 1</code></td> A reference back to the introducer node.</td></tr> 3</td> IntLiteral</code></td> 5</code></td> The raw value 5.</td></tr> 4</td> BinaryOp</code></td> Add(2, 3)</code></td> Adds the NameRef and the Literal.</td></tr> 5</td> Bind</code></td> name: "b", val: 4</code></td> Introducer: "b" exists, bound to Index 4.</td></tr> </tbody></table> The Stack Returns (For Compilation)</h3> In order to deal with this data, we traverse it but we will also want to build up a stack. You might ask: If we flattened the tree, why do we need a stack? While the storage is flat, the compilation process requires a stack to handle scopes. Because let declarations can be nested, we cannot simply scan linearly and remember everything forever. We need to handle when names go out of scope (shadowing). Let's add BlockStart</code> and BlockEnd</code> nodes that indicate nested blocks. // Source Code let x = 10; // Outer 'x' { let x = 20; // Inner 'x' (shadows outer) print(x); // Should print 20 } print(x); // Should print 10 </code></pre> The resolve_names Algorithm</h3> I am too lazy for full code examples, just the idea. We use a SymbolTableStack during the resolution pass. We iterate through the array once. We maintain a stack of scopes, where each scope maps a string name to an integer index. function resolve_names(nodes): # A stack of scopes. Each scope is a Map: String -> Index scope_stack = [ new Map() ] for i, node in enumerate(nodes): match node.kind: case BlockStart: # Push a new, empty scope onto the stack scope_stack.push( new Map() ) case BlockEnd: # Pop the top scope. Inner variables are forgotten. scope_stack.pop() case Bind(name, value_index): # Register the variable in the CURRENT (top) scope. current_scope = scope_stack.top() current_scope.set(name, i) case NameRef(name): # Look for the name, starting from the top scope down. target_index = find_in_stack(scope_stack, name) # PATCH THE NODE: # The node no longer holds "x". It holds the index (e.g., 4). node.resolved_index = target_index </code></pre> After this pass, the stack is discarded. The IR is now "wired." Every variable usage points directly to the instruction that created it. When representing source as AST, we would use an algebraic data type. One could use mutable data structures there, or build up a symbol table. 3. Breaking the Line: Control Flow</h2> So far, execution has been linear: Index 0, then 1, then 2. But branching constructs like if, else, and while break this line. In a tree-based AST, an If node has children pointers to "Then" and "Else" blocks. In our flat array, we may prefer to have in the same contiguous vector, instead of blocks floating in separate memory. So we introduce Jump nodes. The Linear Layout</h3> Consider this source: if (a) { print(1); } else { print(2); } print(3); </code></pre> Here is the Indexed RPN layout. Note the use of BrFalse</code> (Branch if False) and Jmp</code> (Unconditional Jump). Index</th> Node Kind</th> Data</th> Explanation</th></tr></thead> 0</td> NameRef</code></td> "a"</code></td> Load variable a</code>.</td></tr> 1</td> BrFalse</code></td> target: 5</code></td> If a</code> is false, jump to Index 5 (Else).</td></tr> 2</td> Int</code></td> 1</code></td> Start of "Then" block.</td></tr> 3</td> Print</code></td> 2</code></td> </td></tr> 4</td> Jmp</code></td> target: 7</code></td> Jump over the "Else" block.</td></tr> 5</td> Int</code></td> 2</code></td> Start of "Else" block (Target of node 1).</td></tr> 6</td> Print</code></td> 5</code></td> </td></tr> 7</td> Int</code></td> 3</code></td> Merge Point. Execution continues here.</td></tr> 8</td> Print</code></td> 7</code></td> </td></tr> </tbody></table> Building It: Backpatching</h3> When we emit the BrFalse instruction at index 1, we haven't written the Else block yet, so we don't know the target index. It is quite straightforward to deal with that: Emit BrFalse with a placeholder target. Save the index.</li> Emit the "Then" block.</li> Emit Jmp with a placeholder target. Save the index.</li> Mark the current index as the start of "Else". Backpatch (update) the BrFalse at index 1.</li> Emit the "Else" block.</li> Mark the current index as the end. Backpatch the Jmp at index 4.</li> </ol> This effectively flattens the logic of the program into a shape that mirrors how hardware executes instructions: predictable, linear memory access with explicit jumps. 4. Towards Interpretation and Code Gen</h2> We have successfully flattened our source code. We have resolved variable names into absolute indices and lowered high-level control flow into jumps. Now comes the reward. The Interpreter: The "Big Switch"</h3> Because our code is a flat array, we can come up with a Virtual Machine (VM) that looks exactly like a hardware CPU: it has an Instruction Pointer (ip) and a big loop. As a reminder — I will never get tired of repeating this — the difference between a virtual machine and abstract machine is that a virtual machine has instructions, whereas an abstract machine has transitions. A translation to a low-level format and a virtual machine plays the role of an interpreter, which provides an implementation of our language. We can also use it to specify the operational semantics, roughly: however you implement this language, it should produce the same result as the reference interpreter. For "real" languages, often specification comes as an afterthought but there are plenty of situations where one really would like to know how a piece of source code is supposed to behave. For example, to find out if the "real" implementation is correct. Somehow, educated people who really should know better can parrot statements like "undefined behavior is all about compiler optimizations" and completely ignore that "undefined behavior" is first and foremost a gap in the specification. Back to our interpreter: we can do something really simple: since we used ANF, (where every node index represents a runtime value), we don't even need a runtime stack for intermediate calculations. We can simply map the nodes array to a parallel values array. A real implementation would not do this, but if we use an interpreter solely to specify behavior, this is sufficient, and we can defer optimizations. Note that since we have already resolved names to indices, instructions like Bind</code> or BlockStart</code> are effectively metadata. The interpreter can simply skip them. function run_vm(nodes): # Holds the runtime result of every node. values = new Array(size=len(nodes)) ip = 0 while ip < len(nodes): node = nodes[ip] match node.kind: case IntLiteral: values[ip] = node.raw_value ip += 1 case Add: # Direct access by index! No stack popping needed. lhs_val = values[node.lhs_index] rhs_val = values[node.rhs_index] values[ip] = lhs_val + rhs_val ip += 1 case BrFalse: if values[node.cond_index] == False: ip = node.target_index # JUMP else: ip += 1 case Jmp: ip = node.target_index # Unconditional JUMP case _: # Skip metadata nodes (Bind, BlockStart, etc.) ip += 1 </code></pre> The Code Generator</h3> We could also do a source-to-source translation and generate C code. The Indexed RPN shines again, because the complexity of the source language is reduced quite a bit. Since instructions are topologically sorted and dependencies are explicit, generating C can be as simple as a single for loop where every node becomes a temporary variable t{i}. This is maybe not a great way to specify what a language means, but a clear implementation advantage of translating to an existing language is that one can build on top of an existing whole compiler, with optimizations, native code generation backends. How exactly the semantics and runtime aspects of the source language and the target language are connected is of course a design choice and can be wildly different. function generate_c_code(nodes): output = StringBuilder() output.append("int main() {\n") for i, node in enumerate(nodes): # 1. Create a label for every instruction so Jumps can find it # e.g., "L_0:", "L_1:", etc. output.append(f"L_{i}: ;\n") # 2. Create a variable name for this node's result # e.g., "t_0", "t_1" var_name = f"t_{i}" match node.kind: case IntLiteral: # int t_0 = 10; output.append(f" int {var_name} = {node.value};\n") case Add: # int t_2 = t_0 + t_1; lhs = f"t_{node.lhs_index}" rhs = f"t_{node.rhs_index}" output.append(f" int {var_name} = {lhs} + {rhs};\n") case Print: # printf("%d\n", t_5); arg = f"t_{node.arg_index}" output.append(f" printf(\"%d\\n\", {arg});\n") case BrFalse: # if (!t_1) goto L_5; cond = f"t_{node.cond_index}" target = f"L_{node.target_index}" output.append(f" if (!{cond}) goto {target};\n") case Jmp: # goto L_7; target = f"L_{node.target_index}" output.append(f" goto {target};\n") output.append(" return 0;\n}") return output.toString() </code></pre> Conclusion</h3> People will always build more languages, especially domain-specific ones. A realistic work-in-progress language that uses indexed RPN is Carbon</a>. By moving from a tree to an Indexed RPN, we replace heap allocations with a single contiguous vector. What was recursive tree-walking of AST can in many cases become index lookups. So there should be a lot less memory-traffic, and when programs get large, memory traffic can have a significant impact on performance. If you are like me and build toy programming language implementations for fun, consider trying this out and see how it works for you! From Operations to Effects: A Journey Through M-Sets 2025-09-25T00:00:00+00:00 This continues a meditation on effects. My last post on the various meanings of effects in the discourse on programming languages (PL) did very little to show how type and effect discipline may be relevant to memory safety in programming languages. Moreover, I encountered a statement I found easy to agree to "before we talk about effects, we need to understand their semantics". This is an interesting challenge in itself. How do we define precisely what a program that performs I/O actually means? When your code says IO::get</code>, how should we describe what is really happening? We can formally answer this by modeling programs as sequences of operations. This journey will take us from a simple log of operations to the powerful idea of M-sets, and finally draw a bridge to a possible type and effect systems approach. The I/O Logbook 📜</h2> Let's start with a simple I/O model. Imagine we only have two operations: reading an integer from an input stream and writing one to an output stream. We can define this set of operations, $OpIO$, as: $$OpIO := \{ \mathtt{get} \} ∪ \{ \mathtt{put}_i \ | \ i \in \mathbb{N} \}$$ Instead of executing these operations immediately, let's first think of a program as simply generating a log or a sequence of these operations. We can represent this sequence as $OpSeq$, which is the set of all possible finite sequences of $OpIO$ operations (in the language of mathematics, this is the free monoid $OpIO^*$). Under this model, the meaning of a program statement is defined by how it transforms this sequence. It's a function from $OpSeq$ to $OpSeq$. let x = get</code> is a function that appends $ \mathtt{get} $ to the sequence.</li> put 42</code> is a function that appends $ \mathtt{put}_{42} $ to the sequence.</li> </ul> A program that reads an integer and then prints 42 simply produce the sequence $[\mathtt{get}, \mathtt{put}_{42}]$. OK, that is maybe not breath-taking, but it is a start. This describes the "runtime effects" of a program (that word again), but how can we see this as a declarative meaning of our program? Enter the M-Set: The Interpreter 🎭</h3> So, we have a sequence of operations. Now what? This is where M-sets come in. An M-set (specifically an $OpSeq$-set in our case) is a mathematical structure that gives meaning to our sequence. You can think of it as a configurable interpreter for our log. The formal definition involves a right action, which specifies what will happen for every possible operation (get, put 42, etc.). The beauty is that we can now assert that we parameterized our program's meaning (the sequence) by an M-set. We can plug in different M-sets to change the program's behavior without changing the program itself: Console M-set: A handler that interprets get</code> by reading from the keyboard and put i</code> by printing to the console.</li> File M-set: A handler that interprets the operations by reading from and writing to a file.</li> Testing M-set: A handler that interprets get</code> by returning a fixed value (e.g., 0) and put i</code> by adding the integer to a test buffer.</li> </ul> The M-set is the semantic domain that brings our abstract sequence to life. Beyond I/O: The Case of Memory 🧠</h3> Why stop with I/O? We can apply the exact same idea to memory management. Let's define a new set of memory operations: $$ OpMem := \{ \mathtt{alloc} \} ∪ \{ \mathtt{free\ n}\ |\ n \in \mathbb{N} \} $$ Here, $\mathtt{alloc}$ represents allocating a new memory location, and $\mathtt{free}\ n$ represents deallocating location $n$. Just like before, a program's interaction with memory can be modeled as a sequence $MemSeq = OpMem^*$. The M-set for this system would be a model of a heap that defines the semantics of allocating and freeing memory. The Leap to Type and Effect Systems ✨</h3> In many cases, tracking the exact sequence of operations is overkill. We often don't need to know the entire history; we just want a summary of what a function might do. This is the leap from concrete sequences to abstract effects. The operations we've been discussing — get, put, alloc, free — can be called (runtime) effects. A type and effect system is a static analysis tool, built into a compiler, that tracks the set of effects a piece of code might trigger. Instead of a concrete sequence like ["alloc", "free 0"], the effect system provides an approximation. It tells us that a function has the potential to perform the effects {alloc, free}. These effects become part of the function's type, giving developers a clear contract about what the function can do. Furthermore, effects can be parameterized. A function type might look like this: fn update(x: &mut T) -> () effect { free(x[*]) } </code></pre> This signature tells us that the update function may free the memory associated with its parameter x. This brings us to a crucial point: effects are an approximation. The free(x[*])</code> effect means the function may release memory associated with elements x[i]</code>, not that it will on every execution path. This conservative approach is essential for guaranteeing safety. And so, we've traveled from a concrete sequence of operations, to a semantic interpreter (the M-set), all the way to an abstract summary of behavior used in modern type and effect systems to help us write safer, more predictable code. Lots of open questions here how information expressed in this language of effects (the annotations on the types) could be used for checking - we cannot talk about runtime values, only refer to them indirectly through expressions that talk about some set of locations. Unrestricted aliasing may make it impossible to decide whether two expressions refer to an overlapping set of memory locations. It does seem like a step towards something interesting. We started with a modest goal: have an answer to the question on the semantics of effects. It looks like we got there: effects (effect annotations) are may-approximations of runtime effects. Three meanings of Effects 2025-09-19T00:00:00+00:00 In the programming languages discourse, the word "effect" is used with different meanings. This can lead to confusion. In research, it is difficult to avoid jargon, but it comes at the price of making it more difficult to make sense of interconnected meanings programming language concepts. Effects - in more than one sense - are interesting today. The software industry is gearing up to move its system programming artifacts from a dark swamp of legacy into the flushing green meadows of memory safety. I argue between the lines of Memory Safety - the missing definition</a>, programming language design is the source and cause of all memory safety problems. Maybe effects can help turn it into a solution. In the following, I will establish a baseline and then enumerate the three specialized meanings for "effects": Type and Effect Systems</li> Notions-of-Computation Effects</li> Effect Handlers (Algebraic Effects)</li> </ul> I am not aware of others meanings. The TL;DR is that type and effect systems come from a very different tradition than computational effects and effect handlers. This is not merely a terminology discussion, but an analytical effort fishing deep in semantics and pragmatics territory. Before we take the plunge, we have to establish a baseline, though. Side Effects</h2> Imperative programming is about manipulation of state (memory, IO, ...) through a sequence of commands. Choosing this way to program has consequences on our ability to define a static semantics. By static semantics, we mean a type system that determines which programs we consider valid. This enable us to reason about properties of programs without having to run them. Imperative programs are composed of "functions" (really: procedures). We may assign a type to a function to describe inputs and output. This type does not contain any information about the state manipulation. If we imagine a function as a transformation process with a "main" direction going from input to output, the interactions that are not part of the output have been called side effects. These functions are not mathematical functions, even if mathematics is what led to computers and "computation". Thinking about this may tempt us to go on a philosophical tangent on epistemology</a> and the Mangle of Practice</a>, but this is for another time and another post. One often encounters "pure" as an adjective for functions that do not have side-effects. This is as if side effects taint the purity of mathematics, and indeed formal reasoning becomes harder in languages with side effects (evaluation order matters). Is it possible to define a static semantics that retains essential information? Type systems are the most widespread and effective way to get static semantics. They are a special form of static analysis that is built into the programming language design. So why not add effects to type system and track more information that way? Type and Effect System</h2> A type and effect system is just that: the enrichment of types with annotations. The first effect system was about memory access: Lucassen, Gifford "Integrating Imperative and Functional Programming"</a> (1986) wrote about tracking reads, writes and allocation of memory: In the language presented in this paper, information about interference and referential transparency is inferred from three orthogonal properties: the ability to allocate and initialize memory locations whose value may be changed,</li> the ability to read the contents of memory locations whose value may be changed (i.e. to observe side-effects), and</li> the ability to unite new values to existing memory locations (i.e. to cause side-effects).</li> </ul> In general it is undecidable whether an expression will allocate, read or write memory locations whose value may be changed. Effect checking is conservative, and classifies an expression as having each of the three properties unless the opposite can be shown. </blockquote> They discuss the role of effect polymorphism in "Polymorphic Effect Systems"</a> (1988) (updated: thanks Colin Gordon for pointing me to the earlier paper). Nielson and Nielson give a wonderful, timeless description in "Type and Effect Systems"</a>, from 1999. The wikipedia page "Effect system"</a> is very congruent with that. This is a good place to remark that in industry it is at present entirely normal that decades pass before ideas from research get taken up. Time for a brief example: imagine we have a toy imperative programming language with a built-in print</code> command. We can define a print</code> effect and tag function types. Thus we gained the ability to distinguish between functions that print and functions that don't print. This would give us ways to enforce a type discipline where certain operations can take functions that are not permitted to print anything. We can immediately see that we then also need to think about propagating effect information (effect-polymorphism). Our mere decision to be tracking effects in the type system leads to more questions: which effects should we track? And what should we check? Marino and Millstein write about a type and effect checking framework "A Generic Type-and-Effect System"</a> that we are pretty free to choose, as long as rules for checking and adjusting satisfy certain monotonicity properties. So it is very easy to state that something is (or should be) an effect. Whether that is valuable has to be demonstrated by the rules that make use of the effect information. The proof of the pudding is in the eating! Modern programming languages give programmers various ways to "extend the language": user-defined data types and operations (bundled as classes in OOP), only providing built-in operations (primitives) for IO, memory allocation. A challenge in type and effect systems outside of minimal research languages is how these facilities for user-defined constructs should be combined with effects. In particular, abstraction depends on the ability to hide information. Notions of Computation</h2> We now shift our perspective completely and embrace the functional programming view. Away with side effects - unless we can capture them in a mathematical framework. Researchers who study programming languages (PL) have a need to describe PL concepts and semantics formally. Many mathematical and logical characterizations have entered the discourse on programming, for example "boolean", "set", "map" for data types. The meaning of programs as a whole, and programming languages as notation for programs has been approached from a mathematics angle. A particular mathematical field that stands out in PL semantics is category theory. Don't run away yet! It is a language for making precise statements about structure-preserving transformations. The goal is to learn about the structures by studying these transformations. In PL theory, one constantly needs to talk about structures (concepts) that exist across different PLs, or can be expressed in many different ways. And thus, product types, sum types (co-product), co- and contravariance and functors found their way into PL discourse. Not only semantics, but also as actual ways to define data structures (algebraic data types). The connection to category theory is often lost in the process, but that has not led to significant problems so far. The $\lambda$-calculus has been used as a vehicle for PL research. Its basic unit computation step (substitution) lends itself so well for compositionality and formal reasoning. This has given rise to functional programming. When researchers assign meaning to programs through mathematical structures (denotational semantics), the idea is that these are widely understood, or understandable. In this vein, a $\lambda$-term would ultimately correspond to a mathematical function of some kind. But not always a mathematical function from the domain of argument types to the domain of the result type. In his 1989 LICS paper "Computational lambda-calculus and monads", Eugenio Moggi challenged the idea that semantics of $\lambda$-terms should correspond to simple mathematical functions. He pointed out the existence of various firmly established notions of computation which are all better explained by something else, and this would pave the way to reason about program equivalence logically. We will not explain monads here, but just the particular "notions of computation" view. It turns out to not be that difficult. In the context of programming, monads are about program composition. Types and Algebra</h3> Today (!) all programmers understand the concept of a generic type. For instance, we can make sense of the phrase "a type $List[T]$ is a list whose elements are of type $T$". Let's call $List$ a type constructor. A type constructor with a single argument $F[T]$ could be considered an operation that takes a type and returns a type. Other operations that constructing types are $\times$ for product types and $+$ for sum types, and 1 for unit (a type that holds no information), and 0 for a type that has no members. Types built from $\times$, $+$, 1 and 0 can be called algebraic types because they look like terms we use in high school algebra. Now, we need to make a mental step away from types in a given programming language, into a world where instead of types talking about specific values in a given language, we have domains of types that talk about all semantics. We continue to call these types, though, because types can be used in the abstract to describe mathematical structures. Semantics go beyond values, we also have to give meaning to the operations. The insight, then, is this: suppose we have a $T$ that represents all semantic values of our programming language, we can think of a "semantic type constructor" $F$ that transform it into a mathematical structure that very accurately describes a notion of computation: $F[T] = 1 + T$ partial computation: either returns a result $T$ or nothing.</li> $F[T] = List[T]$ non determinism: returns a whole list (or set) of results.</li> $F[T] = S \times T$ stateful computation: operations may read and modify a state $S$</li> </ul> One can think of others. When we see this, we can say $F$ captures the computational effects of the operations given in such a notion of computation. Where in a Rust or Haskell program, you return Some(x)</code> or None</code>, we can lift this to whole computational model or partial computations that either return something or they don't. Such programs (functions) can be composed in a way that preserves their type structure. The "monad" part comes in when we talk about composing program fragments from building blocks. As promised, we will not go into the monad laws and join,unit,bind operations here. It is enough to say: when we focus on the way programs are composed / sequences, we end up with an abstract framework for reasoning about programs with effects. This turns out to be super-useful for practical programming. This approach lets us define our own programming language inside a given programming language. And what's more, we can see embedded "languages" (programs composed from commands) in this way even if the people who built them never heard about monads. Effects in Functional Programming</h3> This is the place to mention Haskell, where the main function of every program is a computation in the IO monad. Over the years, people have taken up this approach in many languages. They: use type constructors in their favorite language to structure their programs to implement domain-specific languages</li> started to call the $F$ itself a "computational effect"</li> write lots of monad tutorials</li> </ul> And this has some justification, because it is useful to define data types and operations that act as domain-specific languages (the "Interpreter" design pattern, anyone?). What if I told you that async</code> is an effect? As we know, a widespread way to implement async</code> functions is to transform them into normal functions that return a Future<T></code>. Future</code> is a unary type constructor and we can come up with rules of composing such functions. I think it is fair to say that the vast majority of programmers will not bother to ask whether monad axioms hold, presumably because the vast majority of programmers will never have the time and energy to learn category theory. So using "effect" as a hand-wavy replacement for monad can certainly be seen in the wild, and we can see the use of it. And what if we want a partial computation returns an error? Result<T, E> = T + E</code> gives us the same ways to compose, even if it is not unary type constructor. This is pretty useful, and we seem not really bothered by the fact that Result</code> takes two type arguments instead of one. This is because in practice, programmers use monads as an "abstraction pattern", mutatis mutandis, and the transformations that would be enabled by adherence to monad laws are less of a concern. Control Effects and Effect handlers</h2> We can see that manipulating memory may be a good candidate for effect tracking, but the print example shows that we can also single out specific operations and consider these as effects. What if we committed to a world of user-definable effects? This is where "algebraic effects" and control flow come in. Control flow is one of the many area where programming language research is looking for something better than mere function calls and for-loops. Consider the ability to suspend computation or return through multiple paths and multiple approaches to package that like exceptions, coroutines, iterators, generators and async functions. The "algebraic" comes in when we consider a user-definable operation and effect, and that operation takes parameters. I will not try to get into definitions but just point to the excellent "Effekt" research programming language</a> where everything is explained pretty well. In particular, a type and effect system is used to check whether the (user-defined) effects are handled. Here, the multiple meanings hit us with full force, the effects - annotations on types - that are being tracked are effects, as in user-defined operations that can be interleave with normal control flow. (updated:) In a language that supports definition of effect handlers one sees the type and effect checking used for a user-definable composition mechanism. In Jonathan Brachthäuser's words: monadic effects start with the semantics (like List[T]</code>) and then see which operations can be supported, while algebraic effects start with the operations (the signature) and then see which domains can support (implement) them. </blockquote> The End</h1> I hope this post helped establish a clearer idea how the venerable idea of type and effect systems, the venerable idea of notions-of-computation and how they get combined in the très en vogue algebraic effects trend in research. There are multiple deep connections between category and programming language technology. We only scratched the surface here. Monads are a remarkable method to structure programming language semantics. That seems to indicate that a science of programming languages could benefit from using the meta language of categories, types and logic. For mere practical uses of a type and effect system, that may not be so relevant. And even in practical uses, we may be fine using a "monad abstraction pattern" for structure embedded, domain-specific programming languages without really being constrained by the "mathematics" that led to the concept. Memory Safety - the missing definition 2025-09-04T00:00:00+00:00 There is still no standard, precise definition of memory safety. Here is a modest proposal. Fact. A programming language (PL) consists of a specification and implementation. </blockquote> The specification lays out precisely what the set of accepted programs is and how the code should behave when the program is executed. An implementation must conform to the specification and is what enables programmers to actually use the PL to translate programs to executable code and execute them. Fact. Programs are executed in an execution environment. </blockquote> An execution environment is not necessarily part of an implementation, but it can be (interpreter). CPU, operating system and system resources are all part of the execution environment. Fact. A specification can, implicitly or explicitly, leave the possibility of execution errors. </blockquote> Definition. There are two classes of execution errors: trapped errors: these are reliably detected and dealt with by the execution environment</li> untrapped errors: these are not detected, and execution goes on. The specification cannot prescribe what behavior take place.</li> </ul> </blockquote> A trapped error may lead to invocation of an error handler or to execution being aborted ("fail-stop"). In contrast, untrapped errors include highly problematic situations like undetected out-of-bounds accesses, use-after-free, integer overflows. They can lead to silent data corruption, hard-to-debug crashes, and also to security vulnerabilities that can be exploited by malicious actors. Definition: Execution Safety A program is execution safe if its execution never leads to untrapped errors. A language L is execution safe if all L-programs are execution-safe. An execution environment is execution safe for a language L if execution of L-programs can never lead to untrapped L-errors. </blockquote> Execution-safety does not guarantee absence of crashes, absence of memory leaks, or possibility to perform modular reasoning on programs. Nevertheless, it is a very strong guarantee and many PLs in widespread use do not have this property. Execution environments can deal with the consequences of untrapped errors, which may limit the harm even if the language used to write these programs may not be execution-safe. Definition: Memory Safety A program is memory safe if no memory access ever leads to untrapped errors. A language L is memory safe if all L programs are memory-safe. An execution environment is memory safe for a language L if execution of L-programs can never lead to untrapped L-errors due to an operation involving memory access. </blockquote> Brief discussion These definitions are directly derived from Luca Cardelli's "Type Systems" article (2004) from the CRC Handbook of Computer Science and Engineering, 2nd Edition, Ch. 97. You can find it. Safety is commonly defined as "absence of unacceptable loss". Turning this into an effective definition in the context of programming requires to clarify what unacceptable loss is. Our definitions above are focused on the mechanics of execution: we do not talk about logic errors, stealing your private key from the .ssh directory or causing crashes. So our definition of memory safety is a "technical definition". Nevertheless, it seems useful: it applies to a range of programming languages, and thus over a range of possible language designs. For safety of execution environment with respoect to a language, it is subtle and important that the definition of error is tied to language spec L. When an implementation translates to a target language of CPU instructions, an operation like "dereferencing a dangling pointer" may turn into "loading memory from an address" which may not be erroneous from the target language point of view. What makes the operation in the execution environment erroneous is the intent. This intent, as far as safety guarantees go, has to be grounded in the language specification. Rust and Carbon, why not both 2025-08-27T00:00:00+00:00 Today's post is a little different. Instead of delving into discussiong of formal type systems, I'd like to step back and talk generally about some aspects of linear type systems as they show up in real life. That means: programming languages and the memory safety discussion today, C++ and Rust. Disclaimer: while the content of this post is very much related to memory safety which is something I work on at Google, this is my personal blog and everything here is my personal opinion. In programming, managing memory and resources is one of the most critical and error-prone tasks. Getting it wrong can lead to nasty bugs, from crashes to security vulnerabilities. Two languages that take this challenge seriously, but with vastly different philosophies, are C++ and Rust. At the heart of this difference lies how their type systems handle the concepts of copying and moving data. While both languages have syntax for these operations, Rust's type system acts as a strict guardian, preventing common errors at compile time, whereas C++ offers more flexibility at the cost of placing the burden of safety squarely on the developer. The C++ Way: A Set of Powerful Conventions</h3> In C++, the ability for an object to be copied or moved is defined by a set of special member functions. If you've written any C++, you've likely encountered the "Rule of Five": Copy Constructor: MyType(const MyType& other);</code></li> Copy Assignment Operator: MyType& operator=(const MyType& other);</code></li> Move Constructor: MyType(MyType&& other);</code></li> Move Assignment Operator: MyType& operator=(MyType&& other);</code></li> Destructor: ~MyType();</code></li> </ul> When you create a type that manages a resource, like a memory buffer, you implement these functions to define what it means to copy or move it. A copy usually involves a "deep copy"—allocating new memory and copying the data over. A move is an optimization that "steals" the internal resource from a temporary object, avoiding a costly allocation. Let's look at a classic example: a simple string-like class. #include <cstring> #include <iostream> class NaiveString { public: NaiveString(const char\* s) { std::cout << "Allocating memory for '" << s << "'\n"; size = std::strlen(s) + 1; data = new char[size]; std::memcpy(data, s, size); } // Move constructor: "steals" the pointer NaiveString(NaiveString&& other) noexcept : data(other.data), size(other.size) { std::cout << "Moving from '" << (other.data ? other.data : "") << "'\n"; // Invalidate the source object! other.data = nullptr; other.size = 0; } ~NaiveString() { if (data) { std::cout << "Freeing memory for '" << data << "'\n"; } else { std::cout << "Destructing a moved-from object\n"; } delete[] data; } char* data; size_t size; }; int main() { NaiveString s1("hello"); std::cout << "s1.data = " << (void\*)s1.data << "\n"; // Move s1 into s2 NaiveString s2 = std::move(s1); std::cout << "s2.data = " << (void\*)s2.data << "\n"; // What about s1 now? std::cout << "s1.data after move = " << (void\*)s1.data << "\n"; // This is the danger zone! C++ lets you do this. // Accessing s1.data here is undefined behavior. // if (s1.data) { // std::cout << "s1 still contains: " << s1.data << "\n"; // CRASH! // } return 0; } </code></pre> The problem here isn't the move itself; it's what happens after. The C++ type system doesn't stop you from trying to use s1 after its resources have been moved to s2. The s1 object is left in a "valid but unspecified state." Accessing it is a landmine waiting to be stepped on. The compiler trusts you not to mess up, but offers no guarantee. The Rust Way: A Contract with the Compiler</h3> Rust builds memory safety directly into its type system through three core principles: Each value has a single owner.</li> There can only be one owner at a time.</li> When the owner goes out of scope, the value is dropped (i.e., its resources are freed).</li> </ol> In Rust, moving is the default behavior for any type that manages resources. When you assign a variable to another, ownership is transferred. fn main() { // String owns heap-allocated memory let s1 = String::from("hello"); // Ownership of the data is MOVED from s1 to s2. // s1 is now considered uninitialized. let s2 = s1; // This line will not compile! // The compiler prevents a "use after move" error. // println!("The value of s1 is: {}", s1); // ^^ value borrowed here after move println!("The value of s2 is: {}", s2); } </code></pre> This is the crucial difference. The Rust compiler tracks ownership. It knows that s1 is no longer valid after the move and turns a potential runtime bug into a compile-time error. It's not a convention; it's a hard-and-fast rule enforced by the type system. So, How Do You Copy in Rust?</h4> What if you have data that is simple, lives entirely on the stack (like an integer), and is cheap to copy? For this, Rust has the Copy trait. A type is Copy if making a bit-for-bit copy of it is a complete and valid clone. Think of types like i32, f64, bool, or a struct containing only other Copy types. When you assign a Copy type, the original variable remains valid. fn main() { // i32 implements the Copy trait let x = 5; // A copy is made. Both x and y are valid. let y = x; println!("x = {}, y = {}", x, y); // This works perfectly! } </code></pre> The compiler enforces a critical rule: you cannot implement Copy for a type that manages a resource. For example, String cannot be Copy because it owns a pointer to heap memory. If you could copy a String, you'd have two variables pointing to the same memory, leading to a "double free" error when both go out of scope. The compiler forbids this, neatly closing another door to memory corruption. The Philosophical Divide 🧐</h3> The difference boils down to two philosophies: C++</th> Rust</th></tr></thead> Trusts the Developer: Provides powerful tools (move semantics) and trusts the programmer to use them correctly. Safety is a matter of discipline and convention (e.g., not using an object after moving from it).</td> Verifies with the Compiler: Bakes ownership and moving directly into the type system. The compiler guarantees at compile time that you cannot use a moved value or accidentally create multiple owners of the same resource.</td></tr> Opt-out Safety: By default, types can be copied or moved if the syntax is valid. The developer must explicitly delete functions (e.g., = delete;) to make a type non-copyable or non-movable.</td> Opt-in Simplicity: Moving is the default for complex types. Simple, bit-wise copying is an opt-in behavior via the Copy trait, which the compiler validates to ensure it's used safely.</td></tr> Runtime Errors: Mistakes like use-after-move or double-frees manifest as crashes or undefined behavior at runtime, which can be difficult to debug.</td> Compile-time Errors: The same mistakes are caught by the compiler before the program can even run, providing clear, actionable feedback.</td></tr> </tbody></table> Under the Hood: A Glimpse into Rust's Type Theory</h3> So how does the Rust compiler pull this off? It's not magic; it's the practical application of ideas from programming language research, specifically a substructural type system. At its core, this system treats variables that own resources as, well, resources. A resource can be used, and once it's fully given away (or "consumed"), it's gone. This is where the idea of a linear type system comes in—a specific flavor of substructural system where a resource must be consumed exactly once. When you write let s2 = s1;, the String owned by s1 is consumed by the move. The type system then invalidates s1 to prevent it from being used a second time, thus upholding the "use once" rule. To enforce this, the compiler uses flow-sensitive typing. This means it doesn't just know a variable's type; it tracks its state as it flows through your code. It knows that s1 is valid before the move, but invalid after it. Finally, this system extends to references (& and &mut), which are more than just pointers. They are special types that come with compiler-enforced invariants, or rules that must always hold true. The most famous of these is that you can have either many immutable references (readers) or exactly one mutable reference (a writer), but never both at the same time. Together, these ingredients—a type system that understands consumption, tracks state through program flow, and enforces strict rules on references—form the foundation of Rust's "borrow checker" and its celebrated compile-time safety guarantees. Bridging the Divide: Interoperability with C++</h3> This brings us to a crucial question: What happens when we interface with existing C++ code? Rust's strict safety discipline, while a huge benefit, can prevent the direct porting of some perfectly valid C++ APIs. In C++, it's common to write APIs that are safe due to specific, documented invariants that a compiler can't necessarily understand. A fantastic example is absl::node_hash_map</code></a>. This data structure guarantees pointer stability: you can look up an element and hold onto a pointer to it, and that pointer will remain valid even if you later mutate the map by adding new elements. This pattern is fundamentally incompatible with Rust's standard aliasing rules. An insert operation on a Rust HashMap requires a unique mutable reference (&mut self), which invalidates all other references to its elements. The Rust compiler has no way to know that Abseil's implementation is special; it sees a violation of its core "aliasing XOR mutability" rule and forbids the code. So, what would it take to teach Rust's type system about the unique guarantees of absl::node_hash_map</code>? You don't. Instead, Rust provides a controlled escape hatch for these situations: the unsafe keyword. To wrap such a library, you would write a safe public API that internally uses unsafe blocks to call the C++ code. The Rust wrapper's creator takes on the responsibility of manually upholding the safety invariants that the borrow checker can no longer verify. This is often accomplished using interior mutability primitives like UnsafeCell<T>. This type tells the compiler, "Don't worry about the borrowing rules for this data; I will manage them myself." By wrapping the C++ map pointer in a type that uses UnsafeCell, you can expose methods like insert that take a shared reference (&self) while still performing mutation internally. The key is that this unsafe logic is contained within a small, well-audited boundary, allowing the rest of the Rust application to interact with it in a completely safe and idiomatic way. The Path Forward: Beyond "Unsafe"</h3> By using unsafe, we've told the Rust compiler to look away, allowing us to manually uphold the invariants of our C++ library. While this violates Rust's standard rules, it's a pragmatic necessity for interoperability. A lot of code will be written this way, and despite this compromise, the resulting programs are still a massive improvement. The vast majority of the application benefits from Rust's static checking, containing the potential for memory bugs to small, auditable sections—a far safer world than a pure C++ program. The problem here isn't a flaw in Rust. Its powerful invariants were simply not designed for the C++ ecosystem, where code may arbitrarily hold on to pointers. The manual reasoning required for unsafe blocks is fragile; a future change elsewhere could unknowingly invalidate the assumptions and reintroduce a bug. This highlights a gap in the programming language landscape: there is no smooth gradient from the "no guarantees" world of C++ to the "full, statically checked" world of safe Rust. This is the gap that experimental projects like Carbon aim to fill. Carbon's goal is to devise a language that enables a gradual migration from C++, combining it with the need for full, statically checked memory safety. To succeed, its type system must find a way to provide guarantees around mutable access even in the presence of aliasing. If Carbon can express a concept like pointer stability directly in its types, it would require a different set of global guarantees and aliasing rules than Rust's. The ultimate challenge lies in safely composing programs written in a "legacy C++" style with new code written in a stricter, safer mode, creating a true bridge to a memory-safe future. Conclusion: Two Paths to a Safer Future</h3> Throughout this journey, we've seen how a language's type system is the bedrock of memory safety. C++ provides the tools for manual resource management, but places the burden of correctness on the developer. Rust, with its revolutionary ownership, move semantics, and Copy</code> trait, shifts that burden to the compiler, turning potential runtime disasters into compile-time errors. While Rust's model is a monumental step forward, its strictness creates a philosophical divide at the boundary with C++. The unsafe</code> keyword offers a pragmatic bridge, but it underscores a fundamental challenge: fitting flexible C++ idioms into Rust's rigid world of invariants. This is precisely why Rust and Carbon should not be seen as competitors, but as different tools for different, equally important jobs. Rust 🚀 excels in new, "greenfield" projects where safety is paramount from day one. It's the ideal choice for building new systems or for creating self-contained, high-performance components that integrate with a larger ecosystem through clear API boundaries compatible with its aliasing rules.</li> Carbon 🏗️ is designed to tackle a different beast entirely: the gradual and interoperable migration of massive, existing C++ codebases. Its goal is to provide an evolutionary path, allowing teams to incrementally bring parts of their code into a world of statically-checked safety without a complete, high-risk rewrite.</li> </ul> Ultimately, both languages are vital to the future of systems programming. They represent a shared mission to eliminate memory bugs, but they approach it from different starting points—one building a new, safe foundation, and the other building a bridge from the world we already have. CEK and Control Operators 2024-01-05T00:00:00+00:00 Today is a good day to recall some foundations of formal abstract machines, continuations, control and data stacks. And a good place to place to start is the CEK machine, described in Felleisen and Friedman's 1986 paper "Control operators, the SECD machine and the $\lambda$ calculus.</a>" and Felleisen's 1987 dissertation</a>. An abstract machine is a device and method for formally specifying program semantics: we define the meaning of a program by describing a state machine that obtains a result. In other words, we are defining the meaning by providing a particular kind of interpreter. This should be fun! An abstract machine does not have instructions but operates on terms. This makes abstract machines different from processors or virtual machines. Terms (abstract syntax) are a high-level, formal representation of a program that fully captures its essence. We will work with a simple $\lambda$-calculus. Formally, $\lambda$-calculus is a rewriting system and the basic question (word problem) consists of deciding whether two terms are equivalent. We want to come up with a mechanical way of simplifying the terms by applying $\beta$ reduction: $$ (\lambda x. M) N \longrightarrow_\beta M [x := N] \ $$ The $\lambda$ calculus is minimalistic but it can nevertheless be used to express all concepts in programming languages. Felleisen's work from that time deals with storage and assignment, too (the CESK machine), but we will not do that here. In order to simplify the exposition, we also defer the discussion of control operators $\mathcal{C}$ or $\mathcal{A}$ Call-by-value evaluation</h2> Let's start with syntax: $$ M, N ::= x\ |\ \lambda x. M\ |\ M N\ $$ A term is either a variable, a $\lambda$-abstraction or an application. In an application $M N$, let's call $M$ the "function" term and $N$ the "argument" term. We ought to discuss free and bound variables, the pitfalls and conventions to avoid them, but we omit this. As mentioned, a computation step is a $\beta$ reduction. Specifically, reduction happens in application terms where the function is a $\lambda$ abstraction. So far this leaves unspecified in which order we should reduce application terms and also what kind of values can be bound to $x$. We want to fix that and clarify that we want a variable to be bound to values $V$, by which we mean $\lambda$-expression that cannot be reduced further, and that we do not reduce under a $\lambda$-abstraction. A way to make this precise is to define evaluation contexts. We describe precisely where an evaluation step can take place and write that particular location as $[~ ]$. $$ {\color{lightgreen}C[~ ]}\ ::= [~ ]\ |\ {\color{lightgreen}C[~ ]}\ N\ |\ V\ {\color{\lightgreen}C[~ ]}\ $$ In other words, we are looking for the innermost application term, and we reduce the function term before we reduce the argument term. It is now interesting and essential to see that an evaluation context ${\color{\lightgreen}C[~ ]}$ can be split up into multiple frames which we can arranged as a list (stack) to recover the original context: $$ \mathrm{Fr} ::= [~ ]\ |\ [~ ] N\ |\ V [~ ] $$ When descending into an evaluation context, we turn each level into a frame and put it on a stack. For the other direction, we take a stack of frames and plug each frame into the next one to obtain the original evaluation context. For example, writing $::$ as separator and having stacks grow to the left, the evaluation context $V_1 (V_2 (([] N)))$ becomes $[~] :: ([~] N) :: (V_2 [~]) :: (V_1 [~])$. Continuations</h2> The key idea now is this: since frames are particular points in the term, we can associate these points with a bit of data that specifies what our abstract machine should do next. The data we need to keep track of are semantic values $V$ which are the results of evaluating expressions and environments $\rho$ which map variables to semantic values. An empty environment is written $\emptyset$ and extending an environment $\rho$ with a binding $x:=V$ is written $[x := V]\rho$ For now, the only kind of semantic value we can have will be a closure, that is, a pair $\langle M, \rho\rangle$ of an abstraction $M$ and an environment $\rho$ that captures assignments for the free variables that might occur. So for example, a $(\lambda x. \lambda y. x y) (\lambda z.z)$ can be evaluated to $\langle \lambda y. x y, [ x := (\lambda z.z) ]\rangle$ Let us call the frame annotated with the data continuation points and write them down. $$ \kappa ::= \mathtt{stop}\ |\ \mathtt{arg}~N\ \rho :: \kappa\ |\ \mathtt{fun}~V :: \kappa $$ The definition reads like the definition of a stack (the data structure) that grows to the left. Compare this to the definition of frames $\mathrm{Fr}$. We do not have a need to represent $[~]$, but we do want to represent the empty stack. We have: $\mathtt{stop}$ for the empty stack,</li> a case $\mathtt{arg}~N~\rho$ for evaluating an argument in environment $\rho$ and</li> a case $\mathtt{fun}~V$ that represents an evaluated function term $V$ waiting for its argument.</li> </ul> We need one more case for returning the result of evaluation. We write this as $\mathtt{ret}\ V$ and note that it can only occur at the top of the stack: $$ K = \mathtt{ret}\ V :: \kappa\ | \ \kappa $$ The CEK machine has a control register C, and environment $\rho$, and a stack K. Together, these three registers form the state. The control register can be empty which we write as $\updownarrow$ or contain a term $M$. The rules are as follows: $$ \begin{array}{lllllll} \langle x, &\rho, &\kappa \rangle &\longrightarrow &\langle \updownarrow, &\emptyset, &\mathtt{ret}~ \rho(x) :: \kappa\rangle \\ \langle \lambda x.M, &\rho, &\kappa \rangle &\longrightarrow &\langle \updownarrow, &\emptyset, &\mathtt{ret}~ \langle\lambda x.M, \rho\rangle :: \kappa \rangle \\ \langle M N, &\rho, &\kappa \rangle &\longrightarrow &\langle M, &\rho, &\mathtt{arg}~ N~ \rho :: \kappa \rangle \\ \langle \updownarrow, &\emptyset, &\mathtt{ret} V :: \mathtt{arg} N \rho :: \kappa \rangle &\longrightarrow &\langle N, &\rho, &\mathtt{fun}~ V :: \kappa \rangle \\ \langle \updownarrow, &\emptyset, &\mathtt{ret} V :: \mathtt{fun} \langle\lambda x. M, \rho\rangle :: \kappa \rangle &\longrightarrow &\langle M, &[x := V]\rho, &\kappa \rangle \\ \end{array} $$ In order to complete the definition we should talk about "loading" $\lambda$-expressions into the machine, let it perform transitions until it is done, then "unloading" the result. Loading $M$ means setting the initial state to $\langle M, \emptyset, \mathtt{stop}\rangle$</li> The terminal state should look like $\langle \updownarrow, \emptyset, \mathtt{ret} V :: \mathtt{stop}\rangle$. From this, we can unload $V$.</li> </ul> There are terms that will make the machine loop. For a closed term $M$, if the machine terminates, it will end up in a terminal state and we should be able to unload. Therefore we can define a partial function $eval_{CEK}$ which loads a term $M$, applies transitions, and if it ends up in a terminal state, returns $V$. One can then prove that if $eval_{CEK} (M) = V$ then $M \longrightarrow^* V$. Control operators</h2> What if we went further in our approach of adding data to our stack frames? Suppose, our semantic values were not only closures but could also capture continuations (stacks). Then we could restore them and get a notion of "jumping". We add two control operators ${\color{lightgreen}\mathcal{C}}$ and ${\color{lightgreen}\mathcal{A}}$: $$ M, N ::= x\ |\ \lambda x. M\ |\ M N\ |\ {\color{lightgreen}\mathcal{C}} M\ |\ {\color{lightgreen}\mathcal{A}} M $$ We skip evaluation context and jump straight to frames: $$ \mathrm{Fr} ::= [~ ]\ |\ [~ ] N\ |\ V [~ ]\ |\ {\color{lightgreen}\mathcal{C}} [~ ]\ |\ {\color{lightgreen}\mathcal{A}} [~ ] $$ Our continuations is almost the same, but gets a continuation marker: $$ \kappa ::= \mathtt{stop}\ |\ {\color{lightgreen}\mathtt{cont}} :: \kappa\ |\ \mathtt{arg} N \rho :: \kappa\ |\ \mathtt{fun} V :: \kappa $$ Our semantic values $V$ can now either be closures or p-continuations $\langle{\color{lightblue}\mathtt{p}}, \kappa\rangle$. With this, we can now add additional transitions for the control operators: $$ \begin{array}{lllllll} \langle {\color{lightgreen}\mathcal{C}} M, &\rho, &\kappa \rangle &\longrightarrow &\langle M, &\rho, &{\color{lightgreen}\mathtt{cont}} :: \kappa\rangle \\ \langle \updownarrow, &\emptyset, &\mathtt{ret} \langle\lambda x. M, \rho\rangle :: {\color{lightgreen}\mathtt{cont}} :: \kappa \rangle &\longrightarrow &\langle M, &[x := \langle{\color{lightblue}\mathtt{p}}, \kappa\rangle]\rho, &\mathtt{stop} \rangle \\ \langle \updownarrow, &\emptyset, &\mathtt{ret} \langle{\color{lightblue}\mathtt{p}}, \kappa_0 \rangle :: {\color{lightgreen}\mathtt{cont}} :: \kappa \rangle &\longrightarrow & \langle \updownarrow, &\emptyset, &\mathtt{ret}\langle{\color{lightblue}\mathtt{p}}, \kappa\rangle :: \kappa_0 \rangle \\ \langle \updownarrow, &\emptyset, &\mathtt{ret}~V :: \mathtt{fun} \langle{\color{lightblue}\mathtt{p}}, \kappa_0 \rangle :: \kappa \rangle &\longrightarrow & \langle \updownarrow, &\emptyset, &\mathtt{ret}~V:: \kappa_0 \rangle \\ \langle {\color{lightgreen}\mathcal{A}} M, &\rho, &\kappa \rangle &\longrightarrow & \langle M, &\rho, &\mathtt{stop} \rangle \\ \end{array} $$ I am not going to reproduce the entire paper. It includes the CK machine (which substitutes eagerly, instead of putting bindings into an environment). This is useful for proving properties, but substitution is something that is avoided in actual implementations. The paper goes on to obtain a calculus (via the CC machine and the C rewriting system) which is proven equivalent to the CEK machine semantics (with control operators). stop: the empty stack</h2> What I wanted for now: a self-contained description of the CEK, a call-by-value abstract machine. When we strip away the discussion of control operators, we can see the beautifully simple idea that capturing "what needs to be done next" in a control stack. The stack is very much a control stack: while is has some data, this data is for the machine itself. The actual argument passing happens in the environment. The environment however, is not actually a stack: it is captured in closures and extended at different times so it looks more like a tree. You can read a bit more in Principles of Programming Languages lecture notes</a> which also extend the CEK wth storage, which yields the CESK. Abstract machines correspond closely to evaluators via transform to continuation passing style. There are many papers about this. Ager, Biernacki, Danvy and Midtgaard A Functional Correspondence between Evaluators and Abstract Machines</a> Abstract machines have been used in correct-by-design programming language implementations. The challenge is in representing environments efficiently: a closure may not need to access the entire environment. There is a nice discussion about the role of abstract machines for PL implementations in Xavier Leroy's talk "From Krivine's machine to the Caml implementations"</a>. A properly efficient implementation would translate to machine code, using registers or generally any calling convention - but if one does use an abstract machine, one can make it better by applying techniques such as keeping some of the environment in the stack and pushing multiple arguments on the stack as a way to access them faster. CBPV and Natural Deduction - Part 4. Polarized Logic 2023-09-16T00:00:00+00:00 This is the last and final part of my little study of polarized natural deduction by means of CBPV. The previous parts are "pt1. small steps"</a>, "pt2. sums and products"</a>, and "pt3. linear logic"</a>. In this part, I will sum up what I find this interesting. This is going to be a bit more personal/opinionated/colored than the previous parts. Proof-theoretical semantics</h2> CBPV as polarized natural deduction helps develop an intuition for "proof-theoretical semantics." It may be a subjective preference, but I consider intuition important. I believe that in the ideal world, intuition, "explanatory power" and teachability would play a role that has weight equal or greater than "new results" of research. This is clearly not the world we live in, but we can nevertheless asipre. So let me draw a rough "baseline" of logic, computation and programming languages. Natural deduction (not sequent calculus) is the standard notation for discussing formal reasoning, proof, structural and substructural proof theory. It goes back to Jaskowski and Gentzen, but today, two people deserve special mention: Dag Prawitz for picking up proof-theoretic semantics and natural deduction, providing a normalization theorem for natural deduction calculi </li> Per Martin-Löf for basing his presentation of intuitionistic type theory</a> on the natural deduction style and establishing the concept of judgment. </li> </ul> </li> Type systems are presented as natural deduction calculi (with "localized assumptions", so a judgment looks like a sequent of sequent calculus). Type systems provide the most effective form of mechanized reasoning and program analysis that we have in programming languages. </li> </ul> Sequent calculus is the device used in more formal studies of reasoning, structured and substructual proof. When we use sequent calculus, we accept a greater distance to intuition and "informal reasoning" for technical reasons. Automated reasoning using "semantic tableaux" is essentially the same as working with sequent calculus. In Proofs and Types</a>, Jean-Yves Girard argues that sequent calculus is "the prettiest illustration of the symmetries of Logic", and that it was "generally ignored by computer scientists." This may have been true at the time, but it is certainly no longer the case. </li> Samson Abramsky does give the sequent calculus its deserved treatment in Computational interpretations of linear logic</a>, though Girard may not have had an operational reading in mind. Abramsky describes a symmetry between constructors (right rules, introduction rules) and destructors (left rules, elimination rules) which gives a proof-theoretic explanation on insights that go back to the McCarthy and Landin. </li> </ul> </li> </ul> Focusing and uniform proofs</h2> Now that we have established a (possibly subjective) baseline, let's talk about focusing. Focusing makes it appearance in logic programming: here computation is not proof normalization, but proof search. Dale Miller, Gopalan Nadathur, Frank Pfenning and Andre Scredrov. "Uniform proofs as the foundation for logic programming". Annals of Pure and Applied Logic. 51:125-157, 1991</li> Around roughly the same time, Jean-Marc Andreoli published "Logic programming with focusing proofs in linear logic.". Journal of Logic and Computation. 2(3):197-347</li> </ul> Focusing is a structuring principle for proofs. It is a way to eliminate the redundancy that is introduced when passing from natural deduction to sequent calculus (Pfennings lectures notes</a>). Girard was aware of Andreoli's work and referenced it "On the unity of logic", introducing "positive" and "negative" polarity as concepts. The key idea is that if a rule of inference is invertible, it makes sense to apply it directly and not look for other rules. When invertible rules have priority over others, this reduces the large search space of proofs. Polarized focusing goes further: in the context of proof search is about "combining runs of connectives that are positive or negative, with explicit coercions between runs. These coercions, written $\uparrow$ and $\downarrow$ are called shift operators." (Pfenning, ibid.) This is what we discussed in this series. What just happened?</h2> We saw how call-by-push-value (CBPV) "separates" typed $\lambda$-calculus into value types and computation types. We obtain a type system which can be viewed as a natural deduction calculus. This view may seem slightly forced, since operations like $\mathtt{thunk}$ and $\mathtt{force}$ that control evalution do not seem very logical. But there are compelling reasons to have them! If we consider the treatment of inference and assumptions in linear logic, the familiar operators $\vee, \wedge$ separate into additive versions $\oplus, \&$ as well as multiplicate versions ⅋ $\otimes$. Some of these types $\oplus, \otimes$ look like data type constructors, and we called these positive or value types. For the others ⅋ $\&$. evaluation does not proceed until the environment demands it. These are negative or computation types. Going back to Bob Harper's post polarity in type theory</a>: positive types come with a single elimination rules which describe how the proof makes sense of the constituent(s) of the proposition. The connective is inductively defined in the sense that the introduction rule completely determines what the elimination rule does. It "writes itself". </li> On the other hand, negative types come with elimination rules that determine what the proofs are going to look like. They are coinductively defined in the sense that "there is no commitment to the internal structure of a proof", anything that provides a way to apply elimination rules is acceptable. These may be called "lazy types", computation is suspended. </li> </ul> There is more to say here (concurrency and "par", applications to memory management, category theory) but that will have to happen some other time. Putting it all together</h2> Types can be defined by either introduction or elimination rules, and the straightforward way to interpret this is to think of negative types as suspending evaluation ("lazy", unevaluated, "objects") and positive types as being fully evaluated ("structured data"). Having both of these in a single calculus, that moreover corresponds to polarized natural deduction, is simply amazing. Surely, being able to encode various formal calculi that are either call-by-name or call-by-value is interesting. From a more practical perspective, it is now easy to see that when one wants to formalize "object-oriented style," which will decidedly involve something like records on unevaluated functions, this will involve negative types. To give a vauge, but hopefully illuminating idea: consider how a "vtable" is quite literally a record of function pointers, "waiting" for program execution to select one among them before proceeding with computation. And how representing an object involves carrying around a vtable. All this makes me think that when formalizing a programming language, it seems very attractive to translate the calculus to CBPV (or something like CBPV). Of course, this supposes that formalization is taking place - it looks like I will finally have some motivation to learn a proof assistant. I hope you found this little series useful in developing intuition for linear logic as well as the role of "negative types." CBPV and Natural Deduction - Part 3. Linear Logic 2023-08-25T00:00:00+00:00 Welcome back to our study of polarized natural deduction and what CBPV has to do with it. For the previous parts of this journey, check out "pt1. small steps"</a> and "pt2. sums and products"</a>. What happened so far</h2> In part 1, we introduced the essential parts of call-by-push-value (CBPV): splitting types into value and computation types, with shifts between these worlds. We saw that static typing constrains evaluation order, without the need for an external specification like evaluation context definition, and that this enables us to give an abstract machine. Although the CK machine still uses substitution, the fact that it only looks at bounded depth from the root makes it already a lot more efficient than a recursive evaluator that has to descend a term and looks for redexes. We hinted at the connection with intermediate representation in compilers, given that all intermediate results are named. In part 2, we added sum and products tyes as value types, and also a second product type as computation type. We hinted that the difference between the two product types has to do with linear logic. In this part, we reveal how linear logic relates to these two different kind of product types. The very notion of propositions having polarity comes from linear logic; going in this direction is a deliberate choice in order to simple, intuitive explanation. Another choice to discuss CBPV is to focus on natural deduction. Almost all linear logic discourse happens on sequent calculi, but Prawitz that natural deduction is illuminating, as witnessed by the fact that the simplest example of Curry-Howard-Lambek correspondence is between $\lambda$ calculus and natural deduction for minimal logic. Let us dive right in and get from natural deduction to linear natural deduction. Assumptions and Linearity</h2> We formulated our typing rules in the standard way, with judgments that read $\Gamma \vdash e : T$. In terms of logic, this is a hypothetical judgment "$T$ is true under assumptions $\Gamma$". This style of writing judgments is called "localized assumptions" or sometimes sequent style: all available assumptions are repeated on every node of our derivation. Most treatments of natural deduction instead assumptions as some leaf nodes that are marked when they are discharged. In the programming view, assumptions are assignments of a type to a variable. In using the $\lambda$-calculus as a programming language, we can use variables in an unconstrained maner, as in these expressions: $\mathrm{twice} := \lambda f: A \rightarrow A. \lambda x: A. f (f x)$ and $\mathrm{double} := \lambda x: Int. x + x$</li> $\mathrm{K} := \lambda x. \lambda y. x$</li> </ul> Type derivations for the bodies $f (f x)$ and $x + x$ and $x$ will use variables from the typing context more than once. In the case of $K$, there will be variable $y$ introduced to the context that is not used at all. These examples show that we use $\Gamma$ as a set of assumptions. We can freely use anything that is in the set. Linearity use a variable exactly once. Just as in mathematics, a linear equation has the shape $a * x + b = 0$ where $x$ is used once (with $a \neq 0$, thus polynomial degree 1), in linear logic, we are interested in the case that an assumption is used exactly once. Why would we do this? There are many concrete applications to linearity in programming languages. The one we will be most interested in is the inherent notion of exclusive ownership that comes from treating types as resources. A close friend of linearity is affinity where polynomial degree may be either 1 or 0. We can ignore variables. This is pretty interesting as you can see from the examples: with an affine type system, we can still assign a type to terms like $K$ combinator that ignore variables. The more practical example is code like this: fn (x: &mut u32) { if (some_condition()) { *x = 1; } } </code></pre> Linear CBPV</h2> We can now define a linear natural deduction calculus, using the standard notation for linear logic connectives. We will do this by simply repeating all the rules we had previously for call-by-push-value, but now with the added constraint that variables (assumptions) are used exactly once. For the judgment $\Gamma \vdash e: T$ we require that exactly the free variables in $e$ appear in $\Gamma$. If we want an affine type discipline instead, we require that $\Gamma$ be a superset of all the variables in $e$. $$ \begin{prooftree} \AxiomC{$x : A^+ \in \Gamma$} \RightLabel{$(\mathrm{hyp})$} \UnaryInfC{$\Gamma \vdash x: A^+$} \end{prooftree} $$ Introduction and elimination rules for linear implication $\multimap$. Note how we use $\Delta$ to emphasize that variables are disjoint from $\Gamma$. $$ \begin{prooftree} \AxiomC{$\Gamma, x: A^+ \vdash s: B^-$} \RightLabel{$(\multimap_I)$} \UnaryInfC{$\Gamma \vdash (\lambda x: A^+. s):\, A^+ \multimap B^-$} \end{prooftree} \quad \quad \begin{prooftree} \AxiomC{$\Gamma \vdash s:\, A^+ \multimap B^-$} \AxiomC{$\Delta \vdash t:\, A^+$} \RightLabel{$(\multimap_E)$} \BinaryInfC{$\Gamma, \Delta \vdash (s\ t):\, B^-$} \end{prooftree} $$ Linear implication $A^+ \multimap B^-$ is different from arrow type: the argument resource $A^+$ is consumed and in return we get a certain computation resource $B^-$. Now comes the shift from values to computations. As before, if you compare with part 1</a>, the only difference is that there is a separate, disjoint context named $\Delta$. $$ \begin{prooftree} \AxiomC{$\Gamma \vdash s: A^+$} \RightLabel{$(\uparrow_I)$} \UnaryInfC{$\Gamma \vdash \mathtt{return}\ s:\, \uparrow\!{}A^+$} \end{prooftree} \quad \quad \begin{prooftree} \AxiomC{$\Gamma \vdash s:\, \uparrow\!{}A^+$} \AxiomC{$\Delta, x: A^+ \vdash t:\, C^-$} \RightLabel{$(\uparrow_E)$} \BinaryInfC{$\Gamma, \Delta \vdash \mathtt{let\ val}\ x = s\ \mathtt{in}\ t:\, C^-$} \end{prooftree} $$ This is easy! The rules for introducing $(\downarrow_I)$ and eliminating $(\downarrow_E)$ the shift from computation to values do not actually change so we don't repeat them here (consult part 1</a>). We have defined a core linear CBPV calculus with abstraction and application. Sums and Products</h2> We move on to sum and product types discussed in part 2</a>. There, we noticed two distinct product types: a value pair type and a computation record type. The sum and product value types correspond to disjunction and conjuction in minimal and intuitionistic logic. In linear logic, there are multiple ways to define disjunction. We use standard, but different symbols for the corresponding linear logic connectives. The linear version of our sum value type correspons to additive disjunction $\oplus$ (still called "plus"): $$ \begin{prooftree} \AxiomC{$\Gamma\vdash v: A_j^+$} \RightLabel{$(\oplus_I)$} \UnaryInfC{$\Gamma \vdash \mathtt{inj}_i\ v : {\color{lightgreen}{{\large \oplus}_I A^+_i}}$} \end{prooftree} $$ $$ \begin{prooftree} \AxiomC{$\Gamma \vdash v : {\color{\lightgreen}{{\large \oplus}_I A^+_i}}$} \AxiomC{$\ldots \Delta_i, x: A_i^+ \vdash M_i : B^- \ldots$} \RightLabel{$(\oplus_E)$} \BinaryInfC{$\Gamma \ldots \Delta_i \ldots \vdash \mathtt{match}\ V\ \{\ldots, \mathtt{case}\ i: x.M_i \ldots\}: B^-$} \end{prooftree} $$ Our pair value type, as hinted at in the last part, is multiplicative conjunction $\otimes$ ("tensor"): we cannot use a component of a pair and drop the other one, we have to use both. $$ \begin{prooftree} \AxiomC{$\Gamma\vdash v_1: A_1^+$} \AxiomC{$\Delta\vdash v_2: A_2^+$} \RightLabel{$(\otimes_I)$} \BinaryInfC{$\Gamma, \Delta \vdash (v_1, v_2) :\, {\color{lightgreen}{A_1^+ \otimes A_2^+}}$} \end{prooftree} $$ $$ \begin{prooftree} \AxiomC{$\Gamma\vdash v: {\color{\lightgreen}{A_1^+ \otimes A_2^+}}$} \AxiomC{$\Delta, x: A_1^+, y: A_2^+ \vdash M: B^-$} \RightLabel{$(\otimes_E)$} \BinaryInfC{$\Gamma, \Delta \vdash \mathtt{match}\ V\ \mathtt{as}\ (x, y).M : B^-$} \end{prooftree} $$ Finally, let us look at the "lazy" (unevaluated) pair computation type. This connective is called additive conjunction and written $\&$ (pronounced "with"). For a linear record of computation, we cannot use multiple fields, we can pick one, losing access to the rest. $$ \begin{prooftree} \AxiomC{$\ldots \Gamma_i \vdash s_i: B_i^- \ldots$} \RightLabel{$({\large \&}_I)$} \UnaryInfC{$\ldots \Gamma_i \ldots \vdash \lambda\{\ldots i.s_i\ldots \}: {\color{lightblue}{{\large \&}_I\ i.B_i^-}}$} \end{prooftree} $$ $$ \begin{prooftree} \AxiomC{$\Gamma\vdash s : {\color{lightblue}{{\large \&}_{i \in I}\ i.B_i^-}}$} \RightLabel{$({\large \&}_E)$} \UnaryInfC{$\Gamma \vdash s\ \mathtt{get}\ i: B_i^-$} \end{prooftree} $$ We have defined a linear CBPV calculus with "plus" $\oplus$ (additive disjunction), "tensor" $\otimes$ (multiplicative conjuction) and "with" $\&$ (additive conjunction) . At this point, you may wonder about the missing combination: multiplicative disjunction. This is written ⅋ (pronounced "par") and corresponds to a sort of separate composition (say, parallel composition of processes). Other notions of composition along two dimension work, too, for instance space and time (fun fact that concurrency is "Nebenläufigkeit" in German, which "side-by-side running"). All this and more linear logic concepts would lead us slightly too far away from our series topic. Towards an understanding of polarity</h2> In our journey to understand polarity in natural deduction, we have arrived at a happy place that lets us see a clear and easy to remember way to relate computation to CBPV, understood as the analogon of an intermediate representation in a compiler. The order of evaluation is specified without the help of external means. In the next and final part, we shall look at polarity in more detail and discuss possible applications for programming language implementation. CBPV and Natural Deduction - Part 2. Sums and Products 2023-08-20T00:00:00+00:00 We are continuing our look at call-by-push-value (CBPV), natural deduction and abstract machines. Last time</a> we looked at a bare-bones version so we could focus on $\lambda$-abstraction and application. In this part, we will only add sum and product types. In the next part we can then shift our view towards linear logic concepts and resources. Sum and Product Types</h2> We add a sum type $ \Sigma A_i $ and a product type $A_1 \times A_2$ to our value types. We also add lazy products, $\Pi_{i \in I}\ i.B_i^-$ for reasons that have to do with the fine-grained distinction between value and computation types. The presentation in this section follows Levy's book closely, except minor changes in syntax. $$ \begin{array}{lll} A^+ &::= & \mathbf{1}\ |\ {\color{lightgreen}{\Sigma_{i \in I} A_i}}\ |\ {\color{lightgreen}{A_1^+ \times A_2^+}}\ |\ \downarrow{A^-} \\ B^- &::= & A^+ \rightarrow B^-\ |\ {\color{lightblue}{\Pi_{i \in I}\ i.B_i^-}}\ |\ \uparrow{A^+} \end{array} $$ By looking at the introduction and elimination rules, we can get intuition what sum types are. can construct value of sum type by injecting, and when we have a value of sum type, we can perform a case-distinction and recover which injector was used. We use a $\mathtt{match}$ syntax for the case distinction. The only special thing for CBPV is that sum types are made from value types, they are themselves value types, but a match is a computation and therefore yields a computation type. $$ \begin{prooftree} \AxiomC{$\Gamma\vdash v: A_j^+$} \RightLabel{$(+_I)$} \UnaryInfC{$\Gamma \vdash \mathtt{inj}_i\ v : {\color{lightgreen}{\Sigma_I A^+_i}}$} \end{prooftree} $$ $$ \begin{prooftree} \AxiomC{$\Gamma \vdash v : {\color{\lightgreen}{\Sigma_I A^+_i}}$} \AxiomC{$\ldots \Gamma, x: A_i^+ \vdash M_i : B^- \ldots$} \RightLabel{$(+_E)$} \BinaryInfC{$\Gamma \vdash \mathtt{match}\ V\ \{\ldots, \mathtt{case}\ i: x.M_i \ldots\}: B^-$} \end{prooftree} $$ We do the same with a pair type: using pair syntax constructs a pair, both components have to be value types and the pair itself is a value type. Elimination is interesting: we require that both values are bound at the same time. The reason for this is that in CBPV, projection would be a computation. $$ \begin{prooftree} \AxiomC{$\Gamma\vdash v_1: A_1^+$} \AxiomC{$\Gamma\vdash v_2: A_2^+$} \RightLabel{$(\times_I)$} \BinaryInfC{$\Gamma \vdash (v_1, v_2) :\, {\color{lightgreen}{A_1^+ \times A_2^+}}$} \end{prooftree} $$ $$ \begin{prooftree} \AxiomC{$\Gamma\vdash v: {\color{\lightgreen}{A_1^+ \times A_2^+}}$} \AxiomC{$\Gamma, x: A_1^+, y: A_2^+ \vdash M: B^-$} \RightLabel{$(\times_E)$} \BinaryInfC{$\Gamma \vdash \mathtt{match}\ V\ \mathtt{as}\ (x, y).M : B^-$} \end{prooftree} $$ Taken together, the value types for sums and products are enough to give us sum-of-product algebraic datatypes, though pattern matching operates only on one level at a time. In terms of logic, sums are disjunction and pairs are conjunction. Defining pairs in this way is a harbinger of what is to follow in the next part: Bob Harper says one might as well write $v_1 \otimes v_2$ (tensor) in his post on polarity in type theory</a>, but admits it is pointless if we are not doing linear logic. We leave this discussion of linear logic to the next part. Now we come to the "lazy product", which is closer in spirit to records of functions (which could be called "objects", but this is simplifying objects a lot). We bundle suspended computations in a record, and we have a computation to select a particular "field" from this record. $$ \begin{prooftree} \AxiomC{$\ldots \Gamma \vdash s_i: B_i^- \ldots$} \RightLabel{$(\Pi_I)$} \UnaryInfC{$\Gamma \vdash \lambda\{\ldots i.s_i\ldots \}: {\color{lightblue}{\Pi_I\ i.B_i^-}}$} \end{prooftree} $$ $$ \begin{prooftree} \AxiomC{$\Gamma\vdash s : {\color{lightblue}{\Pi_{i \in I}\ i.B_i^-}}$} \RightLabel{$(\Pi_E)$} \UnaryInfC{$\Gamma \vdash s\ \mathtt{get}\ i: B_i^-$} \end{prooftree} $$ Also here, we shall leave the corresponding linear logic connector, & "with", for the next part. Machine Transitions</h2> We extend the CK machine accordingly. There is a new stack frame $(\_ \mathtt{get}\ i)$, otherwise the transitions are as one would expect them. As before, the heavy lifting in CK machine is done by substitution. $$ \begin{array}{llll} C & K & \rightsquigarrow & C' & K' \\ s\ \mathtt{get}\ i & k & & s & (\_ \mathtt{get}\ i)::k \\ \lambda\{\ldots i.s_i\ldots \} & (\_ \mathtt{get}\ i)::k & & s_i & k \\ \mathtt{match} (v_1, v_2)\ \mathtt{as}\ (x, y).M & k & & M[x := v_1, y:= v_2] & k \\ \mathtt{match} (\mathtt{inj}\ i\ v) \{ \ldots \mathtt{case}\ i: x.M_i\ldots \} & k & & M_i [x:=v] & k \\ \end{array} $$ Preview of the next part</h2> We have discussed CBPV following a narrative of polarized natural deduction. There were some previews of linear logic discussion, but fundamentally, the types could not yet be interpreted as linear logic proposition. The reason for this is that in natural deduction, assumptions can be used multiple times and discharged whenever we want. This corresponds to the structural rules of sequent calculus, of which Girard says they are the most important of them all. $$ \begin{prooftree} \AxiomC{$x : A^+ \in \Gamma$} \RightLabel{$(\mathrm{hyp})$} \UnaryInfC{$\Gamma \vdash x: A^+$} \end{prooftree} $$ Therefore, if we want to set up linear natural deduction, we have to first make precise where and how this unconstrained use of hypotheses takes place. We can then see how to change perspective and decree that assumptions can be used exactly once (linear) or up to once (affine). Stay tuned for the next session. CBPV and Natural Deduction - Part 1. Small steps 2023-07-22T00:00:00+00:00 I have been reading up on call-by-push-value (CBPV), a variation of $\lambda$-calculus that is more fine-grained in its treatment of argument-passing. I then came across a chapter from Frank Pfenning's 2016 lecture notes on substructual logic</a> that characterizes CBPV as polarized natural deduction. In a small series of posts, I am exploring what this means and how this relates to compilers. The Curry-Howard-Lambek correspondence is a nice way to connect logic and computation. I have written about this in previous posts on my old blog, see Higher-order logic and equality</a> and Intuitionistic Propositional Logic and Natural Deduction</a>. For the first part, those posts should be enough to tag along. In the next parts, we may look at some fine points of natural deduction. Reduction and strategy</h1> In $\lambda$-calculus, our elementary computation step is a $\beta$-reduction. Taking the perspective of the Curry-Howard-Lambek correspondence, we can regard a type as a logical proposition and a term of typed $\lambda$-calculus as a proof of this proposition. Here, $\beta$-reduction is a rewriting of the proof, which removes a 'detour' in the proof. In other words, computation is proof normalization. $$ ( \lambda x : A . M)\ N \longrightarrow_β M [ x := N ] $$ Here, $[ x := N ]$ is our way of writing substitution. The full definition requires the usual careful treatment of bound names. The rewriting can take place anywhere in the term. Suppose we wanted to specify the order of computation steps. When there are not one, but several reducible expressions, which reduction should happen first? A small-step semantics would specify not only reduction, but also where exactly evaluation takes place. To this end, one defines evaluation context, a term with a hole that specifies exactly where in the term reduction is permitted to happen. Here is a sample definition, where we consider $\lambda$-abstractions as values (we do not reduce under a $\lambda$.) $$ \begin{array}{lll} E &::= &x\ |\ \lambda x. E\ |\ E\ E \\ C &::= &[~]\ |\ E\ C\ |\ C\ V \\ V &::= &\lambda x. E \end{array} $$ When we have a context, we can fill its hole with a $\lambda$-expression and now exactly where evaluation takes place. The hole of a context could also be filled with another context. The definition of evaluation contexts forces the argument (operand) in an application to be evaluated before the operator. The example above is thus a form of call-by-value. Note that the definition of $C$ comes with a somewhat arbitrary decision to evaluate the operand before the operator. For call-by-name, passing a term that is not value would require a context like $[~]\ E$. Our specification of evaluation contexts seems to impose order by defining what form of argument-passing is possible; in other words, constraining (or leaving unconstrained) what can be bound to an identifier. Values and computations</h1> Instead of defining evaluation contexts separately, we now look at a different way of impose order on evaluation. In doing so, we start treating contexts as computation (filling the hole with a value is something that can produce a new value). We will see that what we are about to do is not very different from specifying an intermediate representation (IR) of a compiler. CBPV is a calculus that encompasses both call-by-value and call-by-name. It achieves this through a fine-grained distinction between terms that are values vs terms that are computations which is enfored by a type disciple. Therefore we will have value types $A^+$ and computation types $B^-$: $$ \begin{array}{lll} B^- &::= & A^+ \rightarrow B^-\ |\ \uparrow{A^+} \\ A^+ &::= &\mathbf{1}\ |\ \downarrow{B^-} \end{array} $$ The type operators $\uparrow{}$ and $\downarrow{}$ are described below. We add $\mathbf{1}$ (the "unit type") as a base type, with only inhabitant $\mathtt{()}$ that we pronounce as "unit". Base types like $\mathtt{Int}$ or $\mathtt{String}$ would also be value types. All variables have value type. $$ \begin{prooftree} \AxiomC{} \RightLabel{$(1_I)$} \UnaryInfC{$\cdot \vdash (): \mathbf{1}$} \end{prooftree} \quad \quad \begin{prooftree} \AxiomC{$x : A^+ \in \Gamma$} \RightLabel{(hyp)} \UnaryInfC{$\Gamma \vdash x: A^+$} \end{prooftree} $$ The arrow type forces arguments to be of value type and the result to be of computation type. What matters most is the interplay of $\lambda$-abstraction and application. Here are the typing rules for these: $$ \begin{prooftree} \AxiomC{$\Gamma, x: A^+ \vdash M: B^-$} \RightLabel{$(\to_I)$} \UnaryInfC{$\Gamma \vdash (\lambda x: A^+. M):\, A^+ \rightarrow B^-$} \end{prooftree} \quad \quad \begin{prooftree} \AxiomC{$\Gamma \vdash M:\, A^+ \rightarrow B^-$} \AxiomC{$\Gamma \vdash V:\, A^+$} \RightLabel{$(\to_E)$} \BinaryInfC{$\Gamma \vdash (M\ V):\, B^-$} \end{prooftree} $$ In words, a $\lambda$-expression has an arrow type which is a computation (negative) type. We can chain abstractions like $\lambda x:X^+. \lambda y:Y^+. M$ for some term $M$ of computation type, but the argument types $X^+, Y^+$ are forced to be be value (positive) types. Application yields a computation type. This is where it may be useful to remember that filling the hole of a context yields something that we can turn into a value (but it is not a value yet). You may have noticed the $I$ and $E$ letters in the rule names. In natural deduction, every logical connective comes with an introduction and elimination rule. Even though we write these like sequents of sequent calculus, there are a few differences. In sequent calculus, there are left- and right-(introduction)-rules, and a computation step corresponds to the removal of detours (lemmas) via cut-elimination. Shifts</h3> As any type discipline, the above rules make sure that certain programs cannot be written anymore. How can we return a value, though? Or write an identity computation? Or pass a function as an argument to another function? Our calculus is not yet complete. First, we need a way to turn a value into a computation. More precisely, we want to turn a term $V: A^+$ of value (positive) type into a term of computation (negative) type. Let us call this operation $\mathtt{return}\ p$. This is a "shift" between value types and computation types and is made explicit using a type operator $\uparrow\!{}A^+$. The notation requires some decoding work since $\uparrow\!{}A^+$ is a computation (negative) type. $$ \begin{prooftree} \AxiomC{$\Gamma \vdash V: A^+$} \RightLabel{$(\uparrow_I)$} \UnaryInfC{$\Gamma \vdash \mathtt{return}\ V:\, \uparrow\!{}A^+$} \end{prooftree} \quad \quad \begin{prooftree} \AxiomC{$\Gamma \vdash s:\, \uparrow\!{}A^+$} \AxiomC{$\Gamma, x: A^+ \vdash M:\, B^-$} \RightLabel{$(\uparrow_E)$} \BinaryInfC{$\Gamma \vdash \mathtt{let\ val}\ x = M\ \mathtt{in}\ N:\, B^-$} \end{prooftree} $$ The corresponding elimination operation takes a suspended computation and yields a value. We use a "$\mathtt{let\ val}$" declaration as source syntax. Note how the type discipline forces the righthand-side to be a suspended computation and how this imposes an order - we need to have the value before continuing. In turning the the right-hand side into a value, we not only know where the actual computation happens; we also bind the result to a local name. This amounts to all intermediary results being named. Next, we want to "package" a computation into a value (suspend the computation). This will let us pass a $\lambda$-abstraction as an argument to another $\lambda$-abstraction. We introduce an operator $\mathtt{thunk}\ t$ that suspends a computation and an operator $\mathtt{force}\ s$ that resumes a suspended computation. $$ \begin{prooftree} \AxiomC{$\Gamma \vdash M: B^-$} \RightLabel{$(\downarrow_I)$} \UnaryInfC{$\Gamma \vdash \mathtt{thunk}\ M:\, \downarrow\!{}B^-$} \end{prooftree} \quad \quad \begin{prooftree} \AxiomC{$\Gamma \vdash V:\, \downarrow\!{}B^-$} \RightLabel{$(\downarrow_E)$} \UnaryInfC{$\Gamma \vdash \mathtt{force}\ V:\, B^-$} \end{prooftree} $$ Again, the shift operator indicates in the type that we have a suspended computation which is a value. Thus, $\downarrow$ shifts a negative to a positive type, and $\downarrow\!A^-$ can be used in all places that require a value type. Example programs</h3> Let's look at examples, starting with the identity combinator of plain $\lambda$-calculus. We want one of these at every type $I_A := \lambda x: A. x$ of type $A \rightarrow A$. In the above polarized $\lambda$-calculus, we get something close enough: for value types, there is: $\mathit{idval}_{A^+} := \lambda x: A^+. \mathtt{return}\,x$</li> for computation types, we have: $\mathit{idcmp}_{B^-} := \lambda x:\,\downarrow{B^-}. \mathtt{force}\ x$</li> </ul> (Exercise: what are the types of these? What kind of "optimization" would remove these?) Next, let us try a combinator $\mathit{twice}_A$ of type $(A \rightarrow A) \rightarrow A \rightarrow A$. It takes a function $f$ and and argument $x$ and applies $f$ to $x$ twice. In our polarized $\lambda$ calculus, we need to use the other shift operation and end up with: $$ \begin{array}{l} \lambda f:\,\downarrow(A^+ \rightarrow\, \uparrow\!A^+). \lambda x:A^+. \\ \mathtt{let\ val}\ y\ =\ (\mathtt{force}\ f)\ x\ \mathtt{in} \\ \mathtt{return}\ (\mathtt{force}\ f)\ y \end{array} $$ Note how the intermediary result has a name $y$. (Exercise: what is the type of this program? If we used an alternative term which didn't use $\mathbf{return}$, what changes?) An interpreter</h1> The logical reading of these typing rules is that we have set up a particular kind of natural deduction calculus. A term that has a type derivation is a proof, and an introduction followed by an elimitation is clearly a "detour". These detours can be removed and these proof normalization steps correspond to computation steps. At this point, we can write out local reductions: $$\begin{array}{ll} \mathtt{let\ val}\ x\ =\ \mathtt{return}\ V\ \mathtt{in} \mathtt{M} &\rightsquigarrow M[x := V] \\ \mathtt{thunk}\ (\mathtt{force}\ M) &\rightsquigarrow M \end{array} $$ This is not what we were after though. Instead, we can define an abstract machine that specifies exactly which rewriting steps to take when. What is special about an abstract machine is that unlike an interpreter that recursively traverses an expression, the machine always operates at a bounded depth from the top. What follows are transitions rules of a CK machine. Here C stands for control and K is a stack of contexts. The source level $\mathbf{let\ val}\ x\ = \_ \ \mathbf{in}\ M$ expression is shortened to $(\_ \ \mathtt{to}\ x. M)$, and an application where we are waiting for the operator to be evaluated is written $(\_\ V)$. This gives a simple operational semantics, although a CK machine is still a rather high-level description since we need to appeal to substitution in the definition. $$ \begin{array}{llll} C & K & \rightsquigarrow & C' & K' \\ \mathtt{let\ val}\ x = M\ \mathtt{in}\ N & k & & M & (\_ \ \mathtt{to}\ x. N) :: k \\ \mathtt{return}\ V & (\_\ \mathtt{to}\ x. M) :: k & & M[x := V] & k \\ \mathtt{force} (\mathtt{thunk}\ M) & k & & M & k \\ M\ V & k & & M & (\_ \ V) :: k \\ \lambda x. M & (\_ \ V) :: k & & M[x := V] & k \end{array} $$ There is a simple idea behind all this which is worth restating: we statically (through the type system) know that every application $(M\ V)$ comes with an operand that is a value. So: whenever we evaluate an application we start by pushing a value (the operand)</li> when we are done with evaluating the operator and obtain a $\lambda$-term, we can pop a value and continue</li> </ul> More precisely, what we push and pop is an application (evaluation context) with hole in the operator place and an value as operand. Even though this looks like call-by-value, this subsumes call-by-name because a suspended computation can be treated as a value. The stack is a list of nested contexts. In a sense, it is dual to an expression; this can be made precise but we won't do this now. A preview of the continuation</h1> We started from a natural deduction calculus, which is used for formal logical reasoning, and ended up with an abstract machine. Unlike rewriting, we have made a step towards a more mechanical, low-level way of normalizing expressions. The fact that the CK machine is still using substitutions makes it look like we are playing a formal game of symbol manipulation, but if we could continue from here towards a CEK machine which replaces substitutions with environments. On the logical side, since we did not discuss products and sums, we are missing conjunction and disjunction. We did not discuss polarization much. We did not explore classical reasoning, negation, sequent calculus. We did not talk about effects yet. CBPV gives us a handle on computational effects, similar to monads but different. It should be obvious how a lean way to specifying evaluation order helps with describing effects. This is a good time to pause and reflect, before we go to the next round and shed light on some of these topics. In the meantime, here are some pointers to learn more: Frank Pfenning's 2016 lecture notes on substructual logic</a> has a chapter on CBPV. It ends not with a CK machine, but a specification of operational semantics in an ordered logic formalism.</li> CBPV is described in the book "Call-by-Push-Value: A Functional Imperative Synthesis" by Paul Blain Levy. A minor difference is that we spell out the application context with the value here while in Levy's stack only the value gets pushed.</li> Matthias Felleisen's lecture notes</a> have a discussion of CK, CEK, CESK machines. The introduction of environments lets us get rid of substitutions and replace them with environment lookups. This is not yet an efficient language implementation, but it closes the gap. There is discussion about treating the environment more like a call stack, including popping unused values. And (for the CESK) there is a discussion on allocating structures.</li> Bob Harper's post on polarity in type theory</a></li> Zena M. Ariola, Aaron Bohannon, Amr Sabry. Sequent calculi and abstract machines</a> has a thorough discussion of natural deduction.</li> Nick Benton, Gavin Bierman, Valeria de Paiva, Martin Hyland. Term Assignment for Intuitionistic Linear Logic</a></li> </ul> about this site and author 2023-06-12T00:00:00+00:00 about me</h3> This is Burak Emir and you are looking at my homepage and blog. I mainly write about programming languages. I live in Zurich, Switzerland with my wife and two daughters, and work at Google as Alchemist of Happiness. My professional life started earlier, somewhere during my PhD in Martin Odersky's group at EPFL Lausanne where I worked on Scala and translating pattern matching. At Google, I worked as engineer and manager on various systems and teams, building backends, distributed systems, middleware. I then started getting back to programming language related things, and a few years later joined the org responsible for compilers and language-related infrastructure. about this site</h3> I am interested in languages, types, logic, systems where one can bridge theory and practice. Occasionally I write things up. You could call it research, but outside any academic institution. This site is a place where I keep some notes. The posts are a way to share what I learnt on programming methods, programming language technology and occasionally some other topics, like knowledge management or cybersecurity. reach out</h3> email: first.last@gmail.com mastodon: @burakemir@discuss.systems</a> twitter: @burakemir (dormant) artifacts 2023-06-12T00:00:00+00:00 Here are some snapshots from the journey (dblp)</a>: Pattern Matching. Burak Emir. Object-oriented pattern matching.</a>. Thesis, EPFL Lausanne, 2007.</li> Burak Emir, Martin Odersky, John Williams. Matching objects with patterns.</a> ECOOP 2007.</li> Burak Emir, Qin Ma, Martin Odersky. Translation Correctness for First-Order Object-Oriented Pattern Matching.</a> APLAS 2007</li> </ul> </li> Constrained Generics (Universal Polymorphism) Burak Emir, Andrew Kennedy, Claudio Russo, Dachuan Yu. Variance and Generalized Constraints for C# generics.</a> ECOOP 2006. I think variance annotations made it to C# 4.0 later.</li> </ul> </li> arrow-logo I did a few programming courses for kids using LOGO and wrote an interpreter arrow-logo</a> that runs in the browser. Some thoughts on educational programming environments for kids</a> on my old blog.</li> </ul> </li> Datalog and Logic Programming. I develop and maintain a language called Mangle</a> and its implementation, open-sourced it (Nov 2022). More people need to know datalog.</li> Mangle Point in Time 01</a></li> Mangle Point in Time 02</a></li> </ul> </li> Odersky Fest 2023</a> </li> </ul> posts on the old blog</h3> The old blog is at bq9.blogspot.com</a>. Some posts you can find over there: (2021) Worked example of automata runs as sheaves</a></li> (2021) What are Commuting Conversions</a></li> (2020)Relating partial evaluation, multi-stage programming and macros</a></li> (2020) Intuitionistic Logic: Intuitionistic Propositional Logic and Natural Deduction</a></li> Kripke semantics and Tableaux for Intuitionistic Logic</a></li></ul></li> (2020) Higher-order Logic: Higher-order Logic and Equality</a></li> Intuitionistic Higher-Order Logic and Equality</a></li></ul></li> (2010) Yacc is of the Living Dead</a></li></ul> </root> First post 2023-06-12T00:00:00+00:00 a fresh start</h3> I had to redo these pages. Using zola</a> now. Rust, lean, a theme I like, mathjax. My old blog posts are still at bq9.blogspot.com</a>. I wanted to like racket and pollen, but kept on forgetting everything absolutely everything. All I want is to keep some standalone note pages, and some posts, with math and code. Goodness. $e^{i\pi }+1=0$ trait Foo { fn foo(&Self) -> Bar } </code></pre> $$ \begin{prooftree} \AxiomC{$P$} \AxiomC{$P\to Q$} \RightLabel{$(\to_E)$} \BinaryInfC{$Q$} \end{prooftree} $$

4. Towards Interpretation and Code Gen</h2>
We have successfully flattened our source code. We have resolved variable names into absolute indices and lowered high-level control flow into jumps. Now comes the reward.</p>

From Operations to Effects: A Journey Through M-Sets

Three meanings of Effects

Memory Safety - the missing definition

Rust and Carbon, why not both

CEK and Control Operators

CBPV and Natural Deduction - Part 4. Polarized Logic

CBPV and Natural Deduction - Part 3. Linear Logic

CBPV and Natural Deduction - Part 2. Sums and Products

CBPV and Natural Deduction - Part 1. Small steps

about this site and author

reach out</h3>
email: first.last@gmail.com</p>
mastodon: @burakemir@discuss.systems</a></p>
twitter: @burakemir (dormant)</p>

artifacts

First post

Building a Wasm Runner with Cloud Hypervisor

Indexed Reverse Polish Notation, an Alternative to AST

Building a Wasm Runner with Cloud Hypervisor

Step 1: The Minimal Kernel</h3> Cloud Hypervisor works best with a kernel tuned for PVH booting (skipping legacy BIOS/UEFI).</p>

Indexed Reverse Polish Notation, an Alternative to AST

4. Towards Interpretation and Code Gen</h2> We have successfully flattened our source code. We have resolved variable names into absolute indices and lowered high-level control flow into jumps. Now comes the reward.</p>

From Operations to Effects: A Journey Through M-Sets

Three meanings of Effects

Memory Safety - the missing definition

Rust and Carbon, why not both

CEK and Control Operators

stop: the empty stack</h2> What I wanted for now: a self-contained description of the CEK, a call-by-value abstract machine. When we strip away the discussion of control operators, we can see the beautifully simple idea that capturing "what needs to be done next" in a control stack.</p>

CBPV and Natural Deduction - Part 4. Polarized Logic

CBPV and Natural Deduction - Part 3. Linear Logic

CBPV and Natural Deduction - Part 2. Sums and Products

CBPV and Natural Deduction - Part 1. Small steps

about this site and author

reach out</h3> email: first.last@gmail.com</p>

artifacts

First post

4. Towards Interpretation and Code Gen</h2>
We have successfully flattened our source code. We have resolved variable names into absolute indices and lowered high-level control flow into jumps. Now comes the reward.</p>

reach out</h3>
email: first.last@gmail.com</p>
mastodon: @burakemir@discuss.systems</a></p>
twitter: @burakemir (dormant)</p>