Bun Integration

Purpose & scope

This spec expands source §5 (Bun adoption path) into a buildable integration design. It defines exactly how a Bun (and, via NAPI, Node) application binds the engine in both deployment shapes the architecture supports:

Path 1 — Embedded in-process

The headline use case. bun:ffi binds the engine's C ABI (02 — Engine Core's engine.h) directly; the engine runs inside the Bun process. No server, no socket, no wire protocol — function-call latency on the hot path.

Path 2 — Server mode (Bun as client)

Run engine-server (Postgres wire protocol, 07). Bun connects with its built-in Bun.sql Postgres client — zero extra dependencies — for multi-client access and tools that expect Postgres.

It also specifies the @twilldb/bun wrapper package over the raw FFI symbols, the NAPI alternative for one package across Bun and Node, prebuilt-binary packaging and distribution, the normative requirements, failure modes, acceptance criteria, and open questions. The C ABI itself is owned by 02 — Engine Core; this page binds to it and MUST NOT redefine it.

Why Bun

The architecture's central claim is that embeddable and storage-disaggregated stop being contradictory once the storage seam is a pluggable backend rather than a server you connect to (source §0/§1). Bun is the host runtime that realizes both halves of that claim from one engine artifact:

• MUST reuse a single engine build across embedded and server stories: bun:ffi links libengine.{dylib,so} in-process; engine-server is the same library wrapped in a listener (source §2.A). No fork in the engine codebase per host.
• SHOULD use the embedded path as the default — it is the fastest route to a working demo with zero infrastructure (source §5 rollout step 1).

Responsibilities & non-goals

Responsibilities

• MUST bind the stable C ABI from engine.h via bun:ffi for the embedded path, and connect via Bun.sql for the server path.
• MUST marshal JS strings to NUL-terminated C strings for every const char* argument and copy out every borrowed const char* return before the owning object advances or is freed.
• MUST map engine status codes and engine_last_error into typed JS errors, distinguishing retryable from terminal failures.
• MUST free every engine-owned resource exactly once — handles via engine_close, results via engine_result_free, statements via engine_finalize.
• SHOULD expose an ergonomic, typed wrapper (@twilldb/bun) so application code never touches raw pointers.
• MAY ship a NAPI addon as an alternative distribution so one package serves Bun and Node.

Non-goals

• MUST NOT define or alter the C ABI — that contract is owned by 02 — Engine Core.
• MUST NOT implement SQL parsing, MVCC, WAL, caching, or storage in the binding layer; the binding is a thin, faithful shim over the engine.
• MUST NOT own the wire protocol, the pooler, or the server lifecycle — those are 07 — Server Mode.
• The WASM build is a separate distribution target, not covered here — see 11 — Deployment Targets.

Two integration paths at a glance

Path 1 — Embedded via bun:ffi

The headline path. bun:ffi's dlopen maps the exported symbols of libengine.${suffix} into JS-callable functions. The connection string passed to engine_open selects the storage backend by scheme; the call sequence is identical for file:// and s3://.

Raw FFI binding

A faithful binding of the lifecycle + one-shot subset of the ABI. Note the use of Bun's suffix so one source file resolves the right platform extension, and cstring for the engine's NUL-terminated const char* returns.

import { dlopen, FFIType, suffix } from "bun:ffi";

// libengine.dylib (darwin) / libengine.so (linux) / libengine.dll (win)
const { symbols: lib, close: unloadLib } = dlopen(`libengine.${suffix}`, {
  // ---- lifecycle ---------------------------------------------------
  // url is a NUL-terminated cstring: "file://./local.db" or
  // "s3://bucket/mydb?region=...". Returns an opaque handle, or null.
  engine_open:    { args: [FFIType.cstring], returns: FFIType.ptr },
  engine_close:   { args: [FFIType.ptr],     returns: FFIType.void },

  // ---- one-shot execution -----------------------------------------
  // DDL/DML, no result set. Returns an EngineStatus code.
  engine_exec:    { args: [FFIType.ptr, FFIType.cstring], returns: FFIType.i32 },
  // buffered query: writes an owned EngineResult* into *out (ptr-to-ptr).
  engine_query:   { args: [FFIType.ptr, FFIType.cstring, FFIType.ptr], returns: FFIType.i32 },

  // ---- prepared statements ----------------------------------------
  engine_prepare: { args: [FFIType.ptr, FFIType.cstring, FFIType.ptr], returns: FFIType.i32 },
  engine_bind:    { args: [FFIType.ptr, FFIType.i32, FFIType.cstring], returns: FFIType.i32 },
  engine_step:    { args: [FFIType.ptr, FFIType.ptr], returns: FFIType.i32 }, // *done out
  engine_finalize:{ args: [FFIType.ptr], returns: FFIType.i32 },
  engine_reset:   { args: [FFIType.ptr], returns: FFIType.i32 },

  // ---- transactions ------------------------------------------------
  engine_begin:   { args: [FFIType.ptr], returns: FFIType.i32 },
  engine_commit:  { args: [FFIType.ptr], returns: FFIType.i32 }, // blocks until WAL durable
  engine_rollback:{ args: [FFIType.ptr], returns: FFIType.i32 },

  // ---- branching ---------------------------------------------------
  // copy-on-write branch at current LSN; NEW handle on a NEW logical DB.
  engine_branch:  { args: [FFIType.ptr, FFIType.cstring], returns: FFIType.ptr },

  // ---- result access (borrowed pointers into the result arena) -----
  engine_result_rows:    { args: [FFIType.ptr], returns: FFIType.i32 },
  engine_result_cols:    { args: [FFIType.ptr], returns: FFIType.i32 },
  engine_result_colname: { args: [FFIType.ptr, FFIType.i32], returns: FFIType.cstring },
  engine_result_value:   { args: [FFIType.ptr, FFIType.i32, FFIType.i32], returns: FFIType.cstring },
  engine_column_value:   { args: [FFIType.ptr, FFIType.i32], returns: FFIType.cstring },

  // ---- errors / metadata ------------------------------------------
  engine_last_error: { args: [FFIType.ptr], returns: FFIType.cstring }, // borrowed
  engine_changes:    { args: [FFIType.ptr], returns: FFIType.i64 },
  engine_last_lsn:   { args: [FFIType.ptr], returns: FFIType.i64 },

  // ---- freeing -----------------------------------------------------
  engine_result_free: { args: [FFIType.ptr], returns: FFIType.void }, // idempotent on null
});

// helper: JS string -> NUL-terminated C string buffer
const cstr = (s: string) => Buffer.from(s + "\0", "utf8");

// open: storage backend chosen purely by the URL scheme.
const h = lib.engine_open(cstr("s3://bucket/mydb?region=us-east-1"));
// const h = lib.engine_open(cstr("file://./local.db"));   // pure-embedded
if (h === null) throw new Error("engine_open failed (unknown scheme or storage error)");

String & buffer marshaling

• MUST pass every const char* argument (URL, SQL, bound value) as a NUL-terminated buffer — a JS string without the trailing \0 will read past its end in C.
• MUST declare engine const char* returns as FFIType.cstring so Bun reads up to the NUL; FFIType.ptr would yield a raw address and require manual length handling.
• MUST encode/decode as UTF-8 on both directions to match the engine's text representation.
• SHOULD reuse a single allocation per call site where hot, but MUST keep argument buffers alive for the entire duration of the FFI call.

Pointer lifetime & ownership

The ABI is opaque-handle and arena based; the binding mirrors that contract exactly (see 02 — Engine Core, ownership table):

Handle (engine_open / engine_branch)

Owned by the binding. Destroyed exactly once via engine_close. A branch handle is a new logical database and is freed independently of its parent.

EngineResult* (out of engine_query)

Owned by the binding; released exactly once with engine_result_free. Every value/colname read from it is borrowed into its arena and becomes dangling the instant the result is freed — the wrapper MUST copy all values into JS before freeing.

Borrowed const char* (values, colnames, errors)

Never freed by the binding. Valid only until the owning object advances or is freed; for engine_column_value, only until the next engine_step/engine_reset/engine_finalize on that statement. Copy out immediately.

Reading a buffered result and the *out pointer

engine_query writes an owned EngineResult* into a caller-provided EngineResult**. In bun:ffi that is a pointer-to-pointer; allocate an 8-byte slot, pass its pointer, then read the engine pointer back out.

import { ptr, read, toArrayBuffer } from "bun:ffi";

const ENGINE_OK = 0; // see EngineStatus in engine.h (02 — Engine Core)

function query(h: Pointer, sql: string): Record<string, string>[] {
  const outSlot = new BigUint64Array(1);          // holds the EngineResult*
  const status  = lib.engine_query(h, cstr(sql), ptr(outSlot));
  if (status !== ENGINE_OK) throw engineError(h, status);

  const result = Number(outSlot[0]) as Pointer;   // owned EngineResult*
  try {
    const rows = lib.engine_result_rows(result);
    const cols = lib.engine_result_cols(result);
    const names: string[] = [];
    for (let c = 0; c < cols; c++) names.push(lib.engine_result_colname(result, c).toString());

    const out: Record<string, string>[] = [];
    for (let r = 0; r < rows; r++) {
      const row: Record<string, string> = {};
      for (let c = 0; c < cols; c++) {
        // borrowed cstring -> copy into JS BEFORE we free the result
        row[names[c]] = lib.engine_result_value(result, r, c).toString();
      }
      out.push(row);
    }
    return out;
  } finally {
    lib.engine_result_free(result);               // exactly once; idempotent on null
  }
}

Error retrieval

function engineError(h: Pointer, status: number): EngineError {
  // engine_last_error returns a BORROWED, thread-local-per-handle cstring.
  const msg = lib.engine_last_error(h).toString();   // copy out immediately
  return new EngineError(status, msg || `engine status ${status}`);
}

• MUST read engine_last_error(h) immediately after a non-OK status and copy it into a JS string before issuing any further call on h (the next call may overwrite it).
• SHOULD map the numeric EngineStatus (not the message text) to retryable/terminal categories, so retry policy never string-matches on prose.

TS wrapper package — @twilldb/bun

Application code SHOULD never touch raw pointers. @twilldb/bun wraps the FFI symbols in an ergonomic, typed API with structured errors and deterministic resource disposal. It maps one-to-one onto the ABI subset above.

Public API surface

// @twilldb/bun
export function open(url: string): Database;

export interface Database extends Disposable {
  exec(sql: string): number;                                  // rows affected (engine_changes)
  query<T = Row>(sql: string, params?: Param[]): T[];          // buffered rows
  prepare<T = Row>(sql: string): Statement<T>;                 // reusable prepared stmt
  transaction<R>(fn: (tx: Database) => R): R;                  // BEGIN/COMMIT/ROLLBACK
  branch(name: string): Database;                             // copy-on-write @ current LSN
  readonly lastLsn: bigint;                                   // engine_last_lsn
  close(): void;                                              // engine_close (idempotent)
  [Symbol.dispose](): void;                                   // = close()
}

export interface Statement<T = Row> extends Disposable {
  all(...params: Param[]): T[];                               // step to ENGINE_DONE, buffer
  get(...params: Param[]): T | undefined;                     // first row or undefined
  run(...params: Param[]): number;                            // rows affected
  [Symbol.dispose](): void;                                   // engine_finalize
}

export type Param = number | bigint | string | Uint8Array | null | number[]; // -> iN/fN/sN/bN/n/vN
export type Row = Record<string, string>;

export class EngineError extends Error {
  readonly status: number;     // EngineStatus code
  readonly retryable: boolean; // CONFLICT / transient STORAGE
}

Intended usage

import { open } from "@twilldb/bun";

// `using` ⇒ db.close() runs deterministically at scope exit (Symbol.dispose)
using db = open("s3://bucket/mydb?region=us-east-1");

db.exec(`CREATE TABLE IF NOT EXISTS notes (id INTEGER PRIMARY KEY, body TEXT)`);

// parameters are tagged and bound positionally (no string interpolation)
db.query("INSERT INTO notes (id, body) VALUES (?, ?)", [1, "hello"]);

const rows = db.query<{ id: string; body: string }>(
  "SELECT id, body FROM notes WHERE id = ?", [1],
);

// prepared statement, reused across iterations; auto-finalized at scope end
using stmt = db.prepare("SELECT body FROM notes WHERE id = ?");
for (const id of [1, 2, 3]) {
  const r = stmt.get(id);
  if (r) console.log(r.body);
}

// transaction: commit on normal return, rollback on throw
db.transaction((tx) => {
  tx.exec("UPDATE notes SET body = 'x' WHERE id = 1");
  tx.exec("INSERT INTO notes (id, body) VALUES (2, 'y')");
});                       // engine_commit() blocks here until the WAL is durable

// branch = instant clone over shared immutable layers (copy-on-write)
using preview = db.branch("pr-1234");
preview.exec("DELETE FROM notes WHERE id = 1");   // diverges from parent; cheap until written

Parameter mapping

JS values map onto the ABI's string-only typed bind encoding (02 — Engine Core, parameter encoding): a one-character type tag prefixes each value.

Error mapping & resource disposal

• MUST throw EngineError (carrying the numeric EngineStatus and the copied engine_last_error message) on every non-OK status; NULL handle/result returns map to a distinct open/query error.
• MUST set retryable from the status code (e.g. ENGINE_ERR_CONFLICT, transient ENGINE_ERR_STORAGE) so callers retry without parsing prose.
• MUST implement Symbol.dispose on Database (→ engine_close) and Statement (→ engine_finalize) so using guarantees release even on throw; results MUST be freed before the method returns.
• MUST roll back the transaction (engine_rollback) when fn in transaction() throws, and re-throw; commit only on normal return.
• SHOULD register a FinalizationRegistry as a backstop for handles that escaped a using scope, while still treating explicit disposal as the contract (GC timing is not durability).
• SHOULD mark a disposed object and return ENGINE_ERR_MISUSE-equivalent on use-after-dispose rather than dereferencing a freed pointer.

NAPI alternative — one package across Bun and Node

The bun:ffi path is Bun-only. To ship a single package usable in both Bun and Node, compile a NAPI (Node-API) native addon over the same engine.h. NAPI is ABI-stable across Node versions and is supported by Bun, so one .node binary per platform/arch serves both runtimes (source §5).

• MAY ship the NAPI addon instead of (or alongside) the raw-FFI package when a single Bun+Node package is required.
• SHOULD keep the public TS surface (open/exec/query/transaction/prepare/branch) identical across the FFI and NAPI builds so callers are runtime-agnostic.
• SHOULD implement the addon over the same engine.h (e.g. via napi-rs) so there is exactly one engine boundary regardless of binding mechanism.

Path 2 — Server mode (Bun as client)

When Bun is just a client, run engine-server (the same library wrapped in a Postgres wire-protocol listener; 07). Because Bun ships a built-in Bun.sql Postgres client, connecting requires zero extra dependencies.

import { SQL } from "bun";

// Connect through the pooler endpoint, not the engine-server directly.
const sql = new SQL("postgres://user@pooler-host:6432/mydb");

const rows = await sql`SELECT id, body FROM notes WHERE id = ${1}`;
// tagged-template params are sent as bind parameters (not interpolated).

await sql.begin(async (tx) => {
  await tx`UPDATE notes SET body = 'x' WHERE id = ${1}`;
  await tx`INSERT INTO notes (id, body) VALUES (${2}, ${'y'})`;
});                                   // COMMIT acked only after WAL is durable

• MUST connect via the Postgres wire protocol exposed by engine-server; no engine-specific client is needed.
• SHOULD place a pooler (PgBouncer / pgcat in transaction mode) in front to absorb serverless connection bursts; point Bun.sql at the pooler endpoint, not the engine directly. See 07 — Server Mode.
• SHOULD use tagged-template parameters (or the explicit parameter form) rather than string interpolation, so values bind as protocol parameters.
• MUST NOT assume multi-writer concurrency per database — the single-writer-per-DB ceiling holds in server mode too (source §4); concurrency scales by many small databases, not many writers per DB.

Packaging & distribution

The embedded path requires the native engine library to be present at runtime for the host's platform/arch. Distribution mirrors the prevailing prebuilt-binary pattern: a thin JS package plus per-platform binary subpackages installed via optional dependencies.

Platform / arch matrix

Suffix handling & resolution

• MUST use Bun's suffix to form the library filename (libengine.${suffix}) so one source path resolves the correct extension per OS.
• MUST resolve the binary from the installed per-platform subpackage for the current process.platform + process.arch, and fail with a clear, actionable error when none matches (naming the missing platform-arch package).
• SHOULD publish each platform binary as an optionalDependencies entry keyed by os/cpu so the package manager installs only the matching one.
• SHOULD embed an ABI version constant in both engine.h and the wrapper and verify it at dlopen time, failing fast on mismatch rather than calling a stale symbol.
• MAY support an TWILLDB_ENGINE_PATH environment override for local builds and unusual deployment layouts.

Install flow

  bun add @twilldb/bun
        │
        ▼
  resolves optionalDependencies by { os, cpu }
        │
        ├─ darwin-arm64  → @twilldb/bun-darwin-arm64 (libengine.dylib)
        ├─ linux-x64     → @twilldb/bun-linux-x64    (libengine.so)
        └─ ...           → (exactly one matches; others skipped)
        │
        ▼
  import { open } from "@twilldb/bun"
        │
        ▼
  dlopen(`libengine.${suffix}`) from the resolved subpackage
        │
        ▼
  verify ABI version constant  →  ready

Configuration

Failure modes & edge cases

Dependencies & existing pieces to start from

C ABI (engine.h)

The stable boundary owned by 02 — Engine Core. This binding is downstream of it; pin the ABI version.

bun:ffi

Bun's built-in FFI module: dlopen, FFIType, suffix, ptr/read. No third-party dependency for the embedded path.

Bun.sql

Bun's built-in Postgres client for the server path. Zero extra deps.

NAPI / napi-rs

For the cross-runtime addon over the same engine.h (Bun + Node).

Pooler

PgBouncer / pgcat (transaction mode) for server-mode bursts — configured in 07 — Server Mode.

Acceptance criteria / definition of done

• MUST open a file:// database via bun:ffi, run exec/query/prepared/transaction/branch, and close cleanly with no leaked handles, results, or statements (validated under a leak check).
• MUST open the same code against an s3:// URL with no source changes and pass the same functional suite (storage chosen by scheme only).
• MUST map every non-OK EngineStatus to a typed EngineError carrying the copied engine_last_error message and a correct retryable flag.
• MUST demonstrate deterministic disposal: using scopes and explicit close()/finalize() release engine resources on both normal return and thrown exceptions; transaction() rolls back on throw.
• MUST fail fast and legibly on ABI mismatch and on a missing platform binary (no stale-symbol call, no wrong-arch load).
• SHOULD connect via Bun.sql through a pooler to engine-server and pass the same functional suite over the wire.
• SHOULD publish prebuilt binaries for at least darwin-arm64, darwin-x64, linux-x64, and linux-arm64, installable via bun add with correct optional-dependency resolution.
• SHOULD keep the public TS surface identical across the FFI and NAPI builds (a runtime-agnostic conformance test).

Open questions & risks

• FFI call overhead vs micro-batching. For very chatty workloads, does per-call FFI trampoline cost warrant a batched submit API (multiple statements per crossing), or does bun:ffi's JIT'd path make it negligible? Quantify against the embedded micro-loop in 09.
• Streaming large results. The wrapper buffers an EngineResult; for large scans, expose the cursor API (engine_step/engine_column_value) as an async iterator? Coordinate with the engine's streaming cursor question in 02.
• GC vs determinism. How aggressively to lean on FinalizationRegistry as a backstop without letting callers treat GC timing as resource management — durability and lifetime must stay explicit.
• NAPI vs FFI as the default. Ship FFI-only first (Bun-native, simplest) and add NAPI later, or lead with NAPI for cross-runtime reach from day one? Affects the packaging story.
• Worker thread safety. Is one engine handle usable across Bun Worker threads, or strictly one handle per thread? The single-writer-per-DB ceiling suggests per-thread handles; confirm against the engine's threading model.
• WASM binding shape. The Workers build cannot use bun:ffi or a .node addon; what is the JS↔WASM call shape and does the @twilldb/bun surface survive unchanged? See 11.

Related specifications