Puffin

The wasm bytecode VM: Puffin in the browser

This document describes how Puffin runs in the browser: a compact bytecode, a small VM for it written in C and compiled to WebAssembly, and — the point of the whole design — puffincc itself, compiled to that bytecode, running on that VM in the tab, compiling and typechecking the editor's programs. It is written for contributors and for anyone curious how a self-hosted compiler ends up in a web page. The bytecode format and instruction set are specified in docs/BYTECODE.md; this document covers the architecture around them.

Design goals, in order:

  1. One semantics. The language is what the compiler says it is, and the browser runs that — not a hand-maintained parallel implementation that needs a bug hunt every time desugar changes. There is no separate web interpreter; the engine in the tab is puffincc.
  2. True self-hosting on the web. A language whose compiler runs everywhere the language runs has no second-class platforms. The compiler-compiling-itself demo runs in a browser tab.
  3. The native contract is sacred. The 61-bit tag scheme, the kind registry, the stdlib manifest, the variadic protocol, and the 309-check golden corpus are the invariants; the VM bends around them, never the reverse.
  4. Boring technology. No dependence on wasm proposals that aren't universally shipped (no wasm-GC, no exception-handling section, no wasm tail calls), and one toolchain the project already uses (clang).

Architecture at a glance

Why a VM at all, and why puffincc on it? Two alternatives define the boundaries. An interpreter written in Puffin and compiled once would still be a parallel semantics (everything after desugar re-decided) and would pay double interpretation — the VM interpreting an interpreter interpreting user code. Compiling user programs directly to wasm founders on GC (linear-memory wasm needs exactly the collector the VM needs, with none of the VM's explicit-root advantages), on control flow (wasm requires structured control flow, and explicate-control emits arbitrary CFGs, so codegen would need a relooper), and on per-run WebAssembly.instantiate latency. A bytecode VM sidesteps all three: unstructured goto is one opcode, and loading a unit is a memcpy. Running the real compiler on that VM means the semantics in the tab are the compiler's semantics — the exact error messages, the exact variadic protocol, the -O1 optimizer behavior — and every language change reaches the web for free.

The bytecode

Value representation: the tag scheme survives wasm32 intact

A Puffin value stays a tagged 64-bit word — pf remains int64_t — on wasm32:

The cost of i64 values on a 32-bit memory is 8 bytes per slot where 4 would often do. Accepted without ceremony: it buys source-identical runtime code and semantics, and the playground's heaps are small.

The cut point: after uncover-locals

The bytecode backend taps the pipeline after uncover-locals. The IR there is (program info (define locals (f params ...) blocks) ...) with blocks a label→tail map over seq/assign/effect/global-set!/return/goto/if-cmp/tail-app, and every variable a named local with function scope, enumerated in the locals set. This maps 1:1 onto a frame-slot bytecode: a local is a slot index, an assign is an instruction, a tail is a branch. Every pass the two native backends share keeps paying rent — tail-call discovery, main untailing, the -O1 loop recovery and blocks cleanup all run before the cut — and diff-ir stays the differential oracle for the shared frontend. (Cutting earlier, at ANF, would re-decide tail positions and control flow in the new backend; cutting later, after allocate-registers, would model an ISA the VM doesn't have.)

One deliberate consequence: limit-functions and the packed-call protocol (papp, packing strictly above 6 args) stay in the frontend unchanged, even though a VM has no six-register limit. The VM's call convention mirrors the native one — ≤6 direct argument slots, extra args packed in a vector, logical arity carried alongside — so pf_collect_rest compiles verbatim and variadic semantics are provably the same code path as native. Elegance loses to parity.

The instruction set

Variable-length encoding: one opcode byte, then operands. Slots are u16; jump targets are byte offsets after linearization; constants live in per-unit pools. The full opcode table with encodings is in docs/BYTECODE.md; the 29 opcodes group as:

Other prims — car, vector-ref, hash-set, all of lib/ — do not get opcodes: they are PRIM calls at every optimization level, same as the native -O0 route. The VM's prim table (src/vm/vm-prims.inc) is generated from the stdlib.rkt manifest by the self-hosted tools/gen-vm-prims.puf (src/gen-vm-prims.rkt is the Racket cross-check), and the backend computes prim ids from the same list — one more derived view in the manifest discipline. If profiling ever justifies it, hot prims can become opcodes one at a time; the seam is the same one the native backends use for open-coding.

Calls, closures, variadics, tail calls

Frames. Frame slots live on a growable value stack in the VM's memory, not the C stack; frame metadata (function index, return ip, result slot) lives on a separate control stack that holds no Puffin values. Deep non-tail recursion grows the stack up to a configurable depth limit (PUFFIN_VM_MAX_DEPTH), so runaway recursion errors the way native builds do rather than OOMing the tab.

Calling convention. The caller stages arguments in contiguous slots of its own frame; CALL copies them into the callee's formal slots. The logical arity operand is the VM register that replaces native r10/x12: variadic callees read it, and packed calls carry arity > 6 with the packed vector in the last argument slot — precisely the native protocol, so COLLECT calls pf_collect_rest after spilling to pf_arg_spill, which compiles unchanged.

Closures keep kind 4 and their layout — header + slot 0 + captured values — but slot 0 holds a function index (a tagged fixnum) instead of a raw code pointer. Raw code pointers must not appear in heap values: units come and go in a REPL session, and a fixnum index is GC-inert. Function indices are unit-relative and biased by the unit's function-table base at dispatch, so a closure made in one unit calls correctly from another.

Proper tail calls are mandatory and structural. TCALL/TCALLI overwrite the current frame and reuse it — constant stack by construction, for all tail calls including mutual recursion and closure calls, exactly matching the native tail-jmp guarantee. This does not depend on the wasm tail-call proposal: the VM loop is a loop; frame reuse is arithmetic. The -O1 loop recovery and blocks cleanup arrive free, since they run before the cut point.

Symbols, gensym, and the unit format

A compiled program is a unit (.pbc): header with a version word, symbol table, string pool, globals table, function table, entry index, and the instruction stream (exact encoding in docs/BYTECODE.md).

Since Puffin strings are byte strings and write-file writes bytes, backends.puf renders this binary format directly — no new I/O machinery in the compiler.

The VM

Implementation language: C, compiled with wasi-sdk

The VM and the runtime are one compilation unit: the dispatch loop calls pf_cons the way core.c does, includes puffin.h, and inherits the tag scheme from the single header that defines it. One toolchain (clang, already the project's assembler/linker), one language for all runtime code. The alternatives were weighed and declined: hand-written WAT is thousands of lines of untyped stack assembly for a component with a GC in it, and Rust would put an FFI boundary between the VM and the C runtime it calls into on every manifest prim — exactly where bugs live.

The wasm builds use wasi-sdk (clang targeting wasm32-wasi) rather than Emscripten: the host surface is small enough that a purpose-built shim beats a generic runtime by hundreds of KB, and the collector below removes the need for Emscripten's Boehm port. Dispatch is a switch loop (computed goto is a possible later upgrade; measured performance below is with the switch).

The same source compiles natively with plain clang. bin/puffin-vm is the primary development target and a supported execution route in its own right — fast-startup portable execution on machines without the assembler toolchain.

The C runtime: hosted, not replaced

The runtime is small and the VM treasures that: core.c plus lib/*.c compile for wasm32 as-is, with these seams:

Native piece wasm build
Boehm (GC_MALLOC, GC_MALLOC_ATOMIC) replaced behind pf_alloc/pf_alloc_atomic/pf_alloc_raw by the collector below — those entries were the whole allocation ABI all along
pf_fatal/pf_error/exit host abort import (error handling section below)
pf_read_int/pf_read_all (scanf/fread on stdin) wasi-libc fd 0 against the shim's stdin buffer — code unchanged
lib/io.c read-file/write-file/system fopen against the shim's in-memory FS (the module file map); system returns the documented web refusal error
open_memstream (value->string) provided by wasi-libc's musl stdio
__attribute__((constructor)) argv capture the loader passes argv explicitly

The rule that matters: the wasm runtime is the same source files. No #ifdef forests — the platform differences live in the allocator module, the host-abort module, and the shim. lib/*.c does not know it is in a browser. pf_print_result, display formatting, error message texts: all identical, because the corpus diff is textual and the corpus is the gate.

GC: safepoint-only mark-sweep in linear memory

The native VM uses Boehm, like every native Puffin program. The wasm build cannot: Boehm's soundness hinges on scanning the stack, and wasm locals are not memory — a value held only in a wasm local is invisible to any scanner. Emscripten's --spill-pointers mitigation spills pointer-typed i32 locals, but Puffin values are i64 tagged words, not pointer-typed, so a pf held in a wasm local across a collection would be a use-after-free. The other tempting exit — representing values as JS objects or wasm-GC structs and letting V8 collect them — abandons the tag scheme and discards every line of lib/*.c: it is a parallel runtime with extra steps.

The VM changes the problem: unlike compiled native code, an interpreter knows where all the values are. The wasm collector (src/vm/wasm/wasm-gc.c, whose header comment is the detailed reference) is:

make -C src/vm gctest builds the native VM through the collector seam (safepoint hooks, no Boehm) for fast iteration on the collector itself.

Errors that don't kill the tab

Native pf_fatal is exit(255) and (error v) is print + exit(1); a browser session must survive both, and the REPL must keep its globals afterward. The mechanism needs neither setjmp nor the wasm exception-handling proposal:

This is strictly better than native semantics require, and it costs one import.

The host shim

wasi-libc wants a WASI-shaped host. Rather than a generic polyfill, web/src/engine/wasi-shim.js implements the handful of syscalls the runtime actually reaches: fd_write (stdout/stderr → onOutput, streamed synchronously), fd_read (stdin buffer), and path_open/fd_read/fd_write against an in-memory FS initialized from the module file map — which is how puffincc's read-file-based module resolution works unchanged in the browser — plus proc_exit, clock_time_get, and stubs that fail loudly.

The compiler backend (-t bytecode)

The backend chain mirrors the native ones in shape, per the standing lockstep discipline (reference implementation in src/, ported in backends.puf; diff-ir takes bytecode as a target and remains the per-pass differential oracle):

Prelude-and-conclusion mostly dissolves: no callee-save areas, no stack probes — what remains is function-table metadata. The full corpus is green through this route at -O0, -O1, and -O2; the -O1 machinery (loop recovery, blocks cleanup, direct calls, fused compare branches) needs nothing bytecode-specific because it all runs before the cut point.

REPL compilation mode

--repl is a driver flag, implemented identically in both compiler sources, under which collect-globals compiles every top-level define — functions included — into a named global cell (function defines become lambda initializers run in source order), late-binds free variables by name instead of erroring, and wraps each top-level expression in an internal result form that select-instructions-bc lowers to the RESULT opcode. reveal-functions then has nothing to reveal, so every top-level call is indirect — which is exactly what makes redefinition retroactive: defining f again stores a new closure in the same cell, and every earlier unit's GGET f sees it. REPL units always compile at -O0 and render as v2 units. Whole-program mode is unaffected (collect-globals keeps erroring on genuinely unbound names at compile time), and unbound cells hold the reserved #<undef> sentinel, so reading one is a runtime error carrying the variable's name.

RESULT renders the value VM-side with the runtime's own value->string and delivers it through the puffin.repl_result host import (natively: one stdout line) — formatting is the runtime's, never the host's.

The web engine

One engine interface

web/src/engine/index.js is the only import the app and its workers use:

run(source, { input, stdin, onOutput, files, entry })
  -> { ok: true, value: string | null } | { ok: false, error: string }
class Session {
  constructor({ input, stdin, onOutput })
  eval(text) -> { ok, results: string[], error? }
}

vm-engine.js implements it over two wasm artifacts:

Prelude injection for whole-program runs happens inside puffincc (main.puf's prelude-inject), not in JS — the browser has no copy of the prelude to drift.

The artifacts (puffin-vm.wasm, puffin-vm-repl.wasm, puffincc.pbc) are build products generated into web/public/ by tools/gen-web-vm.sh, from the self-hosted build/puffincc — the web pipeline is Racket-free end to end.

The pipeline visualizer

Pipeline mode's provenance (back edges, breadcrumbs) is built on Racket-side object identity: a weak eq-hash tagging every constructed node, composed across passes. puffincc cannot reproduce this cheaply — the bootstrap deliberately dropped prov wrappers, and Puffin hashes are equal?-keyed with no weak references. So, honestly: Pipeline mode is served by src/ir-server.rkt, a dev-mode tool that requires a local Racket process. A provenance-free "layers only" in-browser trace from puffincc's serialize-ir machinery is a recorded idea, not a design.

Measured performance

Numbers below were measured on an Apple M4 Pro (macOS 26.5), node v23.9.0 for the wasm rows, wasi-sdk 33 (-O2, switch dispatch), everything built from the tree (make -C src/vm wasm wasm-repl, tools/gen-web-vm.sh). Native rows are whole-process wall time, best of 5; wasm rows time _start on a fresh instance of a pre-compiled module.

Artifact sizes:

Artifact Raw gzip -9
puffin-vm.wasm (command) 374,954 B 127,236 B
puffin-vm-repl.wasm (reactor) 368,488 B 125,504 B
puffincc.pbc 1,439,866 B 527,407 B
Total added ~2.18 MB ~780 KB

Honesty note: the original design budgeted ≤350 KB raw for the VM wasm and ≤1 MB raw for puffincc.pbc; the shipped artifacts exceed both (7% and 40% over respectively), and the reactor build is a second artifact the budget never priced in. Gzip is what the wire sees, and everything is lazy-loaded on first Run.

Interpretation factor (the same .pbc on bin/puffin-vm and under node through the shim, against native -O1 compiled output):

Workload native -O1 native VM wasm VM (node) factors
fib(30) 6.7 ms 45 ms 72 ms 6.7× / 10.7×
fib(35) 40 ms 460 ms 820 ms 11.5× / 20×
tail-loop 20M adds 45 ms 317 ms 593 ms 7.0× / 13.2×
HAMT 200k insert+lookup 109 ms 125 ms 1,309 ms 1.15× / 12×

So: 7–12× native VM and 13–20× wasm VM slower than native -O1 on compute-bound code — the classic no-JIT bytecode-VM range. Prim-dominated work (the HAMT row) is nearly native speed on the native VM because the time is in the C runtime, not the dispatch loop.

Compile latency in the browser-equivalent path (puffincc on the wasm VM, compile to /out.pbc): hello-world 2 ms, a small typed program 3–4 ms, fib 33 ms, the HAMT program 6 ms. Compilation is prim/allocation-bound (reader, HAMTs, strings), which the VM executes as native C, so the compiler's effective interpretation factor is 1.2–2×, not 13–20×. End-to-end run() (compile + instantiate + execute) under node: hello 2–3 ms, fib(30) **100 ms** — of which ~72 ms is running fib(30) itself.

REPL (engine Session over the reactor build): first-ever boot including wasm module compile ~165 ms; session boot + first eval with warm modules 2–4 ms; per-eval 1.5–8 ms for one-liners through a 100k-iteration loop.

GC effect: a 40-eval allocation-heavy REPL session holds flat at 16 MB where the never-freeing scaffold allocator it replaced grew 16 → 252 MB; a 4M-pair build/consume benchmark halved (783 → ~390 ms), and the full wasm corpus run dropped from ~34 s to 14.3 s — freelist reuse beats never-freeing malloc.

If performance ever needs more, the levers are, in order: open-code the hot prims as opcodes, superinstructions for the MOV-heavy patterns select emits, and slot-reuse frames — all inside the VM and backend, none touching semantics.

Verification

The corpus — 309 golden checks — is the gate on every route, and golden texts are never edited to accommodate the VM: output differences are VM bugs by definition.

Limitations and non-goals