Puffin

The Puffin bytecode (.pbc), versions 1 and 2

This document is the format contract for Puffin's bytecode: the lockstep surface between the bytecode backends — puffincc's (puffincc-src/backends.puf, the primary implementation) and the Racket oracle's (src/backend-bytecode.rkt) — and the VM (src/vm/puffin-vm.c, built as bin/puffin-vm). It is written for contributors touching either side of that surface. Any change to an encoding, an opcode, or the unit layout bumps the version word, and the VM refuses versions it does not implement. The format is not a stable distribution contract: the compiler and the VM ship together; nothing archives .pbc files.

Version 1 is a whole-program unit. Version 2 is a REPL unit (--repl): identical layout except the globals section carries the cells' NAMES (§3.1) and the RESULT opcode may appear. v1 bytes are unchanged by v2's existence. The wasm builds of this VM are what run the browser playground: puffincc itself, compiled to bytecode, executes on the wasm VM, and the playground REPL is a session of v2 units on the reactor build.

0. Quick use

build/puffincc -t bytecode prog.puf -o prog.pbc   # compile a unit
build/puffincc --repl e.puf -o e.pbc     # compile one REPL eval (v2)
build/puffincc --repl-prelude -o p.pbc   # the prelude as a REPL unit
bin/puffin -t bytecode prog.puf          # Racket driver: compile and run on the VM
bin/puffin -c -t bytecode prog.puf       # ... or just produce prog.pbc
racket src/main.rkt -f --repl -o e.pbc e.puf   # Racket oracle, one REPL eval
bin/puffin-vm prog.pbc                   # run a unit
bin/puffin-vm -d prog.pbc                # disassemble a unit
bin/puffin-vm --session u1.pbc u2.pbc    # one SESSION, many units (REPL semantics)
make -C src/vm                           # build bin/puffin-vm (native)
make -C src/vm wasm wasm-repl            # the wasm command + reactor builds
racket src/test.rkt -m bytecode -O 0     # the golden corpus through this route

1. The machine, in one paragraph

A register (frame-slot) machine cut after uncover-locals — the same cut point the native backends use (docs/WASM-VM.md describes the pipeline): each function's named locals become numbered frame slots, each blocks-IR statement becomes one instruction, each tail becomes a branch. Values are the native runtime's 61-bit tagged words, byte for byte (src/runtime/puffin.h: fixnum n<<3, heap ptr|1, immediates #f=2 #t=10 void=18 '()=26, symbol id<<3|3). The VM hosts the existing C runtime — prims are direct calls into libpuffin.a; allocation is Boehm in the native build, and the wasm builds use the VM's own mark-sweep collector (src/vm/wasm/wasm-gc.c, §6) — and frames live on a chunked value stack in GC-visible memory, not the C stack, which is what makes proper tail calls a memmove.

2. Frames, slots, and the call protocol

A frame is nlocals slots of 8 bytes each. Slot layout, fixed by the backend (allocate-slots-bc):

slot 0 .. nformals-1     the formals, in order (arguments arrive here)
...                      remaining named locals, sorted by name
...                      the staging area: call arguments and
                         literal materialization (see §4)

nlocals is always at least 6, so any call site may deliver up to six physical argument values before the callee looks at them. Fresh frames are zeroed (slot value fixnum 0), matching the native zero-seeded globals discipline and keeping the value stack clean for conservative GC.

The call protocol mirrors native exactly — parity with the native calling convention beats VM-local elegance, because closures and variadic collection share C code with native builds:

Closures are kind-4 heap records with the native layout — header, slot 0, captured values — except slot 0 holds a function index as a tagged fixnum (FUNREF), never a raw code pointer: units come and go, and a fixnum is GC-inert and serializable. Call sites read slot 0 with UGET (the frontend already extracts it: lift-lambdas emits (app (unsafe-vector-ref clo 0) clo args...)), so CALLI receives the extracted index value, checks it is a fixnum in range, and dispatches through the function table. Raw code addresses never appear in heap values.

Errors: arity is not checked at calls (native doesn't); CALLI of a value that is not a function index is a fatal error ("application of a non-procedure"). The arithmetic and comparison intrinsics tag-check their operands (§4). Frame-stack depth is capped (default 4M frames; PUFFIN_VM_MAX_DEPTH overrides) so runaway recursion errors like native instead of consuming the machine.

3. The unit format

Little-endian throughout. All strings are length-prefixed byte strings (u32 len + bytes; embedded NULs legal in the string pool).

header    'P' 'U' 'F' 0x01            4 bytes of magic
          u32 version                 1 = whole-program, 2 = REPL unit
          u32 reserved                0
symtab    u32 n, then n names         every symbol literal in the unit
strtab    u32 n, then n byte strings  the string-lit pool
globals   u32 count                   v1: count only (private array)
          [v2 only] count names       cell names, index order (§3.1)
funcs     u32 n, then per function:
            name                      length-prefixed (diagnostics only)
            u32 nformals
            u8  variadic (0/1)
            u8  kfixed                fixed-formal count if variadic
            u16 pad (0)
            u32 nlocals               total frame slots (>= 6)
            u32 code-offset           into the code section
            u32 code-length
entry     u32 function index of main
code      u32 total-length, then the instruction stream

Loading a unit: symbols intern at load. The compiler assigns symbol ids from its sorted literal table, but ids are unit-relative; the loader interns each symtab name through pf_intern_symbol (which dedups) and maps SYM operands to VM-global tagged symbols. Two units loaded into one session therefore agree on eq? for symbols — the same promise the separate-compilation section of docs/MODULES.md makes for native builds, by the same mechanism. String literals materialize lazily, once per literal per unit (the pf_string_const cache behavior, kept: repeated evaluation of one literal is eq?). v1 globals are zero-seeded (fixnum 0), like the native .space array.

One VM instance may load MANY units (--session natively; the wasm reactor build in the browser). Each unit's functions append to one global function table: FUNREF/CALL/TCALL operands are unit-relative and biased by the unit's func_base at dispatch, so the tagged function index in a closure's slot 0 stays valid across units — a closure made in unit A calls correctly from unit B. CALLI/TCALLI dispatch on the global index directly.

A v2 (REPL) unit's globals section carries the cells' names. At load, each name resolves against the session cell table (name → one pf cell, a GC root), creating cells on demand seeded with the reserved unbound sentinel #<undef> (immediate 34, after #f=2 #t=10 void=18 '()=26). GGET of an #<undef> cell is a runtime error carrying the variable's name; GSET stores through the shared cell, which is how redefinition replaces and cross-eval references resolve at call time. v1 units keep their private zero-seeded arrays and pay no check.

The wasm reactor build (make -C src/vm wasm-repl, -DPVM_REACTOR -mexec-model=reactor) exports the session boundary: pvm_boot() once, then pvm_alloc(len) + pvm_load_run(ptr, len) per unit, all in one instance. Aborts (pf_error/pf_fatal) unwind only the wasm frames; the host restores the exported __stack_pointer and the session (heap, interned symbols, cells) survives — pvm_load_run resets the VM frame stacks on entry. This is the build behind the playground REPL (web/src/engine/vm-engine.js drives it).

4. The instruction set (29 opcodes)

Operands: d/a/b/s/fs are u16 frame-slot indices, base a u16 slot index (first staged argument), n a u8 physical argument count (<= 6), arity a u16 logical arity, off a u32 byte offset from the start of the function's code, k64 an i64 tagged word, k8 an i8 tagged word (sign-extended), sym/str/fn/g/prim u16 table indices, i a u8 payload-slot index.

Literals never appear as instruction operands: the backend stages them into staging slots with IMM/SYM first, so every operand is a slot. (The resulting MOV-heavy pattern is a known compression seam — superinstructions — left deliberately unexploited.)

op encoding semantics
0x01 MOV d s R[d] = R[s]
0x02 IMM d k64 R[d] = k64 (any tagged word)
0x03 IMM8 d k8 R[d] = sext(k8) (small fixnums, #f/#t/void/'())
0x04 SYM d sym R[d] = interned symbol for unit symbol sym
0x05 STR d str R[d] = cached heap string for literal str (lazy)
0x06 FUNREF d fn R[d] = fix(fn) (a closure-slot-0 value)
0x07 NEG d a R[d] = 0 - R[a] (tagged, wrapping)
0x08 ADD d a b R[d] = R[a] + R[b] (tagged, wrapping)
0x09 MUL d a b R[d] = (R[a] >> 3) * R[b] (tagged, wrapping)
0x0A LT d a b R[d] = R[a] <s R[b] ? #t : #f (signed tagged)
0x0B EQ d a b R[d] = R[a] == R[b] ? #t : #f
0x0C JMP off ip = off
0x0D BREQ a b off if R[a] == R[b]: ip = off
0x0E BRLT a b off if R[a] <s R[b]: ip = off
0x0F BRLE a b off if R[a] <=s R[b]: ip = off
0x10 BRGT a b off if R[a] >s R[b]: ip = off
0x11 BRGE a b off if R[a] >=s R[b]: ip = off
0x12 CALL d fn base n arity direct call; result to R[d]
0x13 CALLI d fs base n arity indirect: R[fs] is a function index
0x14 TCALL fn base n arity direct tail call: frame reuse
0x15 TCALLI fs base n arity indirect tail call: frame reuse
0x16 PRIM d prim base n manifest prim: direct C call via the table
0x17 COLLECT d k variadic prologue: spill slots 0–5 to pf_arg_spill, R[d] = pf_collect_rest(fix(k), arity)
0x18 UGET d a i R[d] = heap(R[a])[1+i] (unsafe-vector-ref)
0x19 USET a i s heap(R[a])[1+i] = R[s] (unsafe-vector-set!)
0x1A GGET d g R[d] = G[g] (v2: through the named cell; #<undef> errors with the name)
0x1B GSET g s G[g] = R[s] (v2: through the named cell)
0x1C RET s return R[s]; RET from the entry frame ends the unit's run
0x1D RESULT s v2 only: if R[s] is not void, render it with the runtime's value->string and deliver it as a REPL result (wasm: the puffin.repl_result host import; native: one stdout line)

Notes:

5. What the backend passes do

The chain mirrors the native backends' shape, and like every other stage it exists twice in lockstep: in puffincc (puffincc-src/backends.puf, the primary implementation) and in the Racket oracle (src/backend-bytecode.rkt). The passes:

-O1 needs nothing new: loop recovery has already turned self tail calls into gotos, blocks cleanup ran, direct (app (fun-ref f) ...) calls become CALL/TCALL, and fused compare branches use the BRcc family. The full golden corpus is green through this route at -O0, -O1, and -O2.

REPL mode (--repl; both compiler sources): collect-globals compiles every top-level define — functions included — into a NAMED cell (function defines become lambda initializers run in source order), late-binds free variables by name, and wraps each top-level expression in the internal #%repl-result prim, which select-instructions-bc lowers to RESULT. reveal-functions then has nothing to reveal (only the synthesized entry exists), so every top-level call is indirect — which is exactly what makes redefinition retroactive. REPL units always compile at -O0 and render as v2.

6. The VM (src/vm/puffin-vm.c)

One C file linking the runtime:

7. Known gaps