Scaffolding CLI & Multi-Client Distribution

Why a scaffolding CLI

The embedded payoff — a persistent, in-process database with zero infrastructure — is only one open() call away, but a new user still has to assemble a project around it: a package.json that depends on the client, a connection string for the right backend, a first table, and a runnable script. twilldb new collapses that into one command, emitting a working starter that mirrors the verified examples under clients/bun/examples/.

Design principles

• MUST be a dependency-free Rust binary in its own crate (crates/cli), hand-rolling argument parsing and placeholder substitution like the rest of the workspace — no clap, no template engine. It only writes files; it never links the engine.
• MUST embed its templates at compile time (include_str!), so the binary is self-contained and works offline — no network fetch, no version skew between binary and templates.
• MUST be language-neutral: the scaffolder generates projects for multiple client ecosystems, so it is not itself published into any one of them as the primary channel.
• MUST NOT overwrite existing files; new refuses a non-empty target directory and both subcommands refuse to clobber an individual file.
• SHOULD mirror the dispatch shape of twill-bench — a thin main.rs over a testable run_cli(&[String]) -> i32 core — so exit codes are asserted without spawning a process.
• SHOULD recognise not-yet-shipped clients and answer with a roadmap-aware message, not "unknown value".
• SHOULD be interactive on a terminal but never block automation: prompt only for fields omitted on the command line, and skip prompting entirely when stdin is not a TTY (falling back to defaults). The wizard is pure over its read/write streams so it is testable without a pseudo-terminal.

Command structure

twilldb new <name> [options]    # create a new project in ./<name>
twilldb init [options]          # scaffold into the current directory
twilldb version                 # print the version
twilldb help                    # usage

# options
-c, --client <bun>              # client ecosystem (default: bun)
-b, --backend <file|s3>         # storage backend / connection string (default: file)
    --vector                    # include a vector-search (HNSW) starter
-y, --yes                       # accept defaults; never prompt

# examples
twilldb new                     # full interactive wizard (on a terminal)
twilldb new notes
twilldb new search --vector
twilldb new app --backend s3
twilldb init

The generated Bun starter is a working @twilldb/bun project: package.json (pinned to the workspace version), tsconfig.json, .gitignore, an app.ts derived from examples/notes.ts, and a README.md. --backend only changes the connection string written into app.ts (file://./<name>.db vs an s3://your-bucket/<name> placeholder) — the engine selects the backend purely by URL scheme, so nothing else differs. --vector adds a vectors.ts starter (a vector(3) column, an HNSW index, a top-k query) and wires a vectors script.

The C ABI is the universal client seam

Only the Bun starter ships today; node, php, and rust are recognised by --client but answer with a roadmap message rather than generating. That is not a placeholder accident — it reflects the architecture. Every language client is a thin FFI shim over the same frozen C ABI (crates/engine/include/engine.h) and the same prebuilt libengine binaries; none reimplements the engine. The Bun client (bun:ffi over the header, with per-platform @twilldb/engine-* binary packages as optional dependencies) is the reference shape each future client copies.

Distribution: two different problems

"How do we distribute this?" has two separate answers, and conflating them is the common mistake. The CLI / server binaries are standalone executables; the client libraries are language packages. They travel different roads.

The CLI binary → a Homebrew tap (and friends)

The CLI is distributed primarily through a Homebrew tap — a self-hosted formula repo (bihaviour/homebrew-twilldb). A tap has no notability/maturity gate and needs no registration, unlike homebrew-core (which also would not accept the project's BUSL-1.1 license, a source-available license rather than an OSI-approved one). Users run:

brew tap bihaviour/twilldb      # → github.com/bihaviour/homebrew-twilldb
brew install twilldb

# or the fully-qualified one-liner (Homebrew needs three parts: user/repo/formula)
brew install bihaviour/twilldb/twilldb

The formula builds from source via cargo (depends_on "rust" => :build), the simplest robust option for v1 — no per-platform binary matrix to maintain. .github/workflows/release-cli.yml keeps the tap current: on every v* tag it downloads the source tarball, computes its sha256, renders Formula/twilldb.rb, and pushes it to the tap repo (using a HOMEBREW_TAP_TOKEN PAT). The canonical formula and setup steps live in packaging/homebrew/.

The client libraries → each ecosystem's own registry

Homebrew is irrelevant for the language clients. A Bun/Node developer installs from npm; a PHP developer from Composer; a Python developer from PyPI. Each client ships through the registry its users already install from — and none of those have a maturity gate either.

The shared release pipeline (the scaling lever)

The piece that makes a polyglot client matrix tractable is building libengine once per platform and fanning the same artifacts out. The project already does the npm half (the @twilldb/engine-{os}-{arch} per-platform packages that @twilldb/bun depends on); every other ecosystem reuses those same binaries.

build libengine.{so,dylib,dll} × {linux,darwin,win} × {x64,arm64}
   ├── npm:       @twilldb/engine-{os}-{arch}   (Bun + Node depend on these)
   ├── Packagist: PHP wrapper loads the matching binary via FFI
   ├── PyPI:      wheels bundling the binary
   └── Homebrew tap / Scoop / cargo / curl|sh:  the CLI + server binaries

Verification — what ships today

Related specifications