Lix is a C++23 implementation of the Nix package manager (~76k lines). The goal is to compile libexpr (the Nix language evaluator) to WebAssembly and drive it from Go via wazero, while keeping the store (path validation, builds, fetching) implemented natively in Go.

Key constraints established during analysis:

A binary codec used for all WASM↔Go data passing. Designed to be simple, length-prefixed (no escaping), and unambiguous.

message   = total_len ":" value

value     = record | variant | scalar
record    = "{" *( klen ":" kbytes "," vlen ":" vbytes ";" ) "}"
variant   = "<" taglen ":" tagbytes value
scalar    = len ":" rawbytes

list(T)   = "{" *( len ":" T ";" ) "}"
pairs(K,V) = "{" *( klen ":" K "," vlen ":" V ";" ) "}"

Type Encoding
StorePath len: + raw path string bytes
bool 1:1 or 1:0
uint64_t 8: + 8 bytes little-endian
Hash n: + raw hash bytes (n = hash size: 16/20/32/64)
HashType 1: + single byte (1=md5, 2=sha1, 3=sha256, 4=sha512)
absent optional key omitted entirely
error {3:err,n:message;} in place of normal fields 

Bla(String)              encodes as   <3:Bla,n:string_bytes
InputAddressed(path)     encodes as   <2:ia43:/nix/store/...
CAFixed(method, hash)    encodes as   <8:ca-fixed{6:method,n:str;4:hash,n:bytes;8:hashType,1:t;}


# isValidPath response (true)
14:{4:bool,1:1;}

# queryPathInfo response
198:{7:narHash,32:<32 bytes>;8:hashType,1:3;7:narSize,8:<8 bytes LE>;4:refs,{43:/nix/store/aaa-glibc;43:/nix/store/bbb-gcc;};6:deriver,43:/nix/store/ccc-gcc.drv;}

# env pairs inside a derivation
3:env,{4:name,5:hello;3:src,43:/nix/store/aaa-src;}

All requests include 3:op,n:opname; as the first field. A single WASM import function lix_store_call dispatches on the op name.

WASM allocates a single buffer via _malloc(req_len + resp_max). First half holds the encoded request, second half receives the encoded response.

lix_store_call(req_ptr, req_len, resp_ptr, resp_max) -> resp_len

Go reads req_len bytes from req_ptr, decodes, executes, encodes response, writes to resp_ptr. Returns actual response byte count (negative = error).

req:  {3:op,11:isValidPath;4:path,n:storepath;}
resp: {4:bool,1:<0|1>;}


req:  {3:op,13:queryPathInfo;4:path,n:storepath;}
resp: {
  7:narHash,n:hash;
  8:hashType,1:t;
  7:narSize,8:u64;
  4:refs,{n:storepath;...};
  6:deriver,n:storepath;    (omitted if absent)
  2:ca,n:ca_string;         (omitted if absent)
}


req:  {3:op,14:readDerivation;4:path,n:storepath;}
resp: <derivation record>


req:  {3:op,15:writeDerivation;<derivation fields>}
resp: {4:path,n:storepath;}


{
  3:op,n:opname;              (in requests only)
  4:name,n:str;
  8:platform,n:str;
  7:builder,n:str;
  4:args,{n:str;...};
  3:env,{n:key,n:val;...};
  10:inputSrcs,{n:storepath;...};
  9:inputDrvs,{n:storepath,{n:outputname;...};...};
  7:outputs,{n:outname,<output>;...};
}

DerivationOutput variants:

InputAddressed:   <2:ia,n:storepath
CAFixed:          <8:ca-fixed{6:method,n:str;4:hash,n:bytes;8:hashType,1:t;}


req:  {3:op,16:computeFSClosure;4:path,n:storepath;}
resp: {n:storepath;...}


req:  {3:op,10:ensurePath;4:path,n:storepath;}
resp: {2:ok,1:1;}


req:  {3:op,14:addTextToStore;4:name,n:str;7:content,n:bytes;4:refs,{n:storepath;...};}
resp: {4:path,n:storepath;}


req:  {3:op,9:fetchTree;n:attrkey,n:attrval;...}
resp: {7:outPath,n:storepath;7:narHash,n:hash;8:hashType,1:t;...}


lix/
  lix/libstore/
    wasm-store.hh         # new: WasmStore class declaration
    wasm-store.cc         # new: WasmStore + codec encode/decode
  lix/libexpr/
    wasm-exports.cc       # new: exported C entry points called by Go
  lix/libstore/meson.build  # modified: add emscripten platform branch
  lix/libutil/signals.cc    # modified: #ifdef __EMSCRIPTEN__ no-op guards
  lix/libexpr/eval.cc       # modified: GC_start_mark_threads guard
  lix/libexpr/primops.cc    # modified: stub prim_exec, prim_importNative
  lix/libexpr/primops/
    fetchTree.cc          # modified: delegate to lix_store_call fetchTree
    fetchMercurial.cc     # modified: delegate to lix_store_call fetchTree

gonix/
  go.mod
  go.sum
  codec/
    codec.go              # bencode-variant encoder/decoder
    codec_test.go         # round-trip tests
  store/
    store.go              # Go Store interface
    types.go              # PathInfo, Derivation, Tree structs
    local.go              # LocalStore: talks to nix daemon socket
  wasm/
    runtime.go            # wazero setup, WASI config, module lifecycle
    imports.go            # lix_store host module, dispatch, memory passing
  cmd/eval/
    main.go               # thin CLI driver

Concrete subclass of Store. Overrides every async virtual method with a synchronous WASM import call. Inherits all pure synchronous methods (parseStorePath, printStorePath, makeOutputPath, computeStorePathForText, etc.) from the base class — these never cross the boundary.

// wasm-store.hh
#pragma once
#ifdef __EMSCRIPTEN__
#include "lix/libstore/store-api.hh"

namespace nix {

struct WasmStoreConfig final : StoreConfig {
    using StoreConfig::StoreConfig;
    const std::string name() override { return "WASM Store"; }
};

struct WasmStore final : Store {
    WasmStoreConfig config_;
    WasmStoreConfig & config() override { return config_; }
    const WasmStoreConfig & config() const override { return config_; }

    explicit WasmStore(WasmStoreConfig config);
    std::string getUri() override { return "wasm://"; }

    // All async virtuals delegate to lix_store_call.
    // Pure sync virtuals (parseStorePath etc.) inherited from Store.

    kj::Promise<Result<bool>> isValidPath(const StorePath & path) override;
    kj::Promise<Result<std::shared_ptr<const ValidPathInfo>>>
        queryPathInfoUncached(const StorePath &, const Activity *) override;
    kj::Promise<Result<StorePath>> addTextToStore(
        std::string_view name, std::string_view s,
        const StorePathSet & refs, RepairFlag repair) override;
    kj::Promise<Result<void>> ensurePath(const StorePath & path) override;
    kj::Promise<Result<StorePathSet>> computeFSClosure(
        const StorePathSet & paths, StorePathSet & out,
        bool flipDirection, bool includeOutputs,
        bool includeDerivers) override;
    // ... remaining virtuals throw Error("not supported in WASM build")

    kj::Promise<Result<std::optional<TrustedFlag>>> isTrustedClient() override {
        return {result::success(Trusted)};
    }
    static std::set<std::string> uriSchemes() { return {"wasm"}; }
    ref<FSAccessor> getFSAccessor() override { unsupported("getFSAccessor"); }
};

} // namespace nix
#endif // __EMSCRIPTEN__

The codec (encode/decode) lives in wasm-store.cc as static helper functions. Each WasmStore method:

  1. Encodes its arguments into a request buffer.
  2. Allocates a response buffer.
  3. Calls lix_store_call(req_ptr, req_len, resp_ptr, resp_max).
  4. Decodes the response.
  5. Returns an immediately-resolved kj::Promise wrapping the result.

The external import declaration:

extern "C" int32_t lix_store_call(
    const char * req_ptr, int32_t req_len,
    char *       resp_ptr, int32_t resp_max_len
);

File Change
libutil/signals.cc checkInterrupt() → no-op; signal handler thread → no-op
libutil/processes.cc runProgram() → throw Error("not supported in WASM")
libexpr/eval.cc:202 GC_start_mark_threads() → #ifndef __EMSCRIPTEN__ guard
libexpr/primops.cc prim_exec, prim_importNative → throw unsupported
libexpr/primops/fetchTree.cc delegate to lix_store_call with op=fetchTree
libexpr/primops/fetchMercurial.cc delegate to lix_store_call with op=fetchTree

Exported C functions called by Go via wazero. Handles are indices into a std::vector<RootValue> with GC_add_roots keeping them alive.

extern "C" {
    void    nix_init(const char * store_dir, int32_t len);
    int32_t nix_eval_expr(const char * expr, int32_t expr_len,
                          char * resp, int32_t resp_max);
    int32_t nix_force_value(int32_t handle, char * resp, int32_t resp_max);
    int32_t nix_get_attr(int32_t handle,
                         const char * name, int32_t name_len,
                         char * resp, int32_t resp_max);
    int32_t nix_get_attr_names(int32_t handle, char * resp, int32_t resp_max);
    int32_t nix_list_length(int32_t handle);
    int32_t nix_list_get(int32_t handle, int32_t idx,
                         char * resp, int32_t resp_max);
    void    nix_free_value(int32_t handle);
    int32_t nix_last_error(char * buf, int32_t max_len);
}

Responses use the same codec format. nix_eval_expr returns a handle (int32 ≥ 0) on success; callers use nix_get_attr, nix_list_get, etc. to traverse the value tree lazily without forcing the entire expression.

libstore/meson.build — add Emscripten branch in the platform chain:

if host_machine.system() == 'linux'
  liblix_sources += files('platform/linux.cc')
elif host_machine.system() == 'darwin'
  liblix_sources += files('platform/darwin.cc')
elif host_machine.system() == 'freebsd'
  liblix_sources += files('platform/freebsd.cc')
elif host_machine.system() == 'emscripten'
  liblix_sources += files('platform/fallback.cc')  # no-op platform
  liblix_sources += files('wasm-store.cc')
else
  liblix_sources += files('platform/fallback.cc')
endif


cd bdwgc
LDFLAGS="-sBINARYEN_EXTRA_PASSES='--spill-pointers'" \
  emconfigure ./configure \
    --enable-threads=none \
    --disable-parallel-mark \
    --host=wasm32-unknown-emscripten
emmake make

Key flags:

em++ -std=c++23 \
  -DHAVE_BOEHMGC=1 \
  -DWASM_STORE=1 \
  -sBINARYEN_EXTRA_PASSES='--spill-pointers' \
  -sSTANDALONE_WASM=1 \
  -fwasm-exceptions \
  -sEXPORTED_FUNCTIONS='["_nix_init","_nix_eval_expr","_nix_force_value","_nix_get_attr","_nix_get_attr_names","_nix_list_length","_nix_list_get","_nix_free_value","_nix_last_error","_malloc","_free"]' \
  [libexpr sources] \
  [libutil sources] \
  [libstore: path.cc derivations.cc content-address.cc store-api.cc wasm-store.cc] \
  -lboehmgc \
  -o libnixexpr.wasm

STANDALONE_WASM=1 produces WASI-compatible output: POSIX file I/O maps to WASI calls, which wazero services via a mounted real filesystem. No JS runtime needed.

Pure Go, no external dependencies. Implements the wire format above.

// Value sum type
type Value interface{ isValue() }
type Record  []Field          // { k,v; k,v; }
type Variant struct {         // <tag payload
    Tag     string
    Payload Value
}
type Scalar  []byte           // len:bytes
type List    []Value          // { v; v; }

type Field struct {
    Key   []byte
    Value []byte
}

func Encode(v Value) []byte        // prefixes with total_len:
func Decode(b []byte) (Value, error)

Both encoder and decoder are iterative (no recursion depth issues for deeply nested derivations). The decoder validates length prefixes before reading to avoid panics on malformed input.

// store.go
type Store interface {
    IsValidPath(ctx context.Context, path string) (bool, error)
    QueryPathInfo(ctx context.Context, path string) (*PathInfo, error)
    ReadDerivation(ctx context.Context, path string) (*Derivation, error)
    ComputeFSClosure(ctx context.Context, path string) ([]string, error)
    EnsurePath(ctx context.Context, path string) error
    AddTextToStore(ctx context.Context, name, content string, refs []string) (string, error)
    WriteDerivation(ctx context.Context, drv *Derivation) (string, error)
    FetchTree(ctx context.Context, attrs map[string]string) (*Tree, error)
}

// types.go
type PathInfo struct {
    NarHash          []byte
    HashType         byte
    NarSize          uint64
    Refs             []string
    Deriver          string   // empty if absent
    CA               string   // empty if absent
}

type DerivationOutput struct {
    Tag      string   // "ia" or "ca-fixed"
    Path     string   // ia: store path
    Method   string   // ca-fixed: method string
    Hash     []byte   // ca-fixed: raw hash bytes
    HashType byte     // ca-fixed: hash type
}

type Derivation struct {
    Name      string
    Platform  string
    Builder   string
    Args      []string
    Env       map[string]string
    InputSrcs []string
    InputDrvs map[string][]string  // path -> output names
    Outputs   map[string]DerivationOutput
}

type Tree struct {
    OutPath  string
    NarHash  []byte
    HashType byte
}

local.go — first-pass implementation shells out to nix CLI:

func (s *LocalStore) IsValidPath(ctx context.Context, path string) (bool, error) {
    cmd := exec.CommandContext(ctx, "nix", "store", "verify", "--no-contents", path)
    err := cmd.Run()
    if err != nil { return false, nil }
    return true, nil
}

This can be replaced with a proper Unix socket implementation of the worker protocol (framing: length-prefixed serialisation over /nix/var/nix/daemon-socket/socket) once the end-to-end path is verified.

type Runtime struct {
    r     wazero.Runtime
    mod   api.Module
    store store.Store
}

func New(ctx context.Context, wasmBytes []byte, s store.Store) (*Runtime, error) {
    r := wazero.NewRuntime(ctx)

    // WASI snapshot preview1 (filesystem, clock, args)
    if _, err := wasi_snapshot_preview1.Instantiate(ctx, r); err != nil {
        return nil, err
    }

    // lix_store host module
    if err := registerStoreImports(ctx, r, s); err != nil {
        return nil, err
    }

    // real filesystem passthrough for .nix file reads
    cfg := wazero.NewModuleConfig().
        WithFSConfig(wazero.NewFSConfig().WithDirMount("/", "/")).
        WithStdout(os.Stdout).
        WithStderr(os.Stderr)

    mod, err := r.InstantiateWithConfig(ctx, wasmBytes, cfg)
    if err != nil {
        return nil, err
    }

    return &Runtime{r: r, mod: mod, store: s}, nil
}

func (rt *Runtime) EvalExpr(ctx context.Context, expr string) (string, error) {
    // allocate buffers in WASM linear memory using exported _malloc
    malloc := rt.mod.ExportedFunction("malloc")
    free   := rt.mod.ExportedFunction("free")
    evalFn := rt.mod.ExportedFunction("nix_eval_expr")
    // ... allocate, call, decode response
}


func registerStoreImports(ctx context.Context, r wazero.Runtime, s store.Store) error {
    return r.NewHostModuleBuilder("lix_store").
        NewFunctionBuilder().
        WithFunc(func(ctx context.Context, m api.Module,
            reqPtr, reqLen, respPtr, respMax uint32) int32 {

            reqBytes, ok := m.Memory().Read(reqPtr, reqLen)
            if !ok {
                return -1
            }

            msg, err := codec.Decode(reqBytes)
            if err != nil {
                return writeError(m, respPtr, respMax, err)
            }

            op := fieldStr(msg, "op")
            respBytes, err := dispatch(ctx, s, op, msg)
            if err != nil {
                respBytes = codec.EncodeError(err)
            }

            if uint32(len(respBytes)) > respMax {
                return -2 // response too large — caller should retry with larger buffer
            }
            m.Memory().Write(respPtr, respBytes)
            return int32(len(respBytes))
        }).
        Export("call").
        Instantiate(ctx)
}

func dispatch(ctx context.Context, s store.Store, op string, msg codec.Value) ([]byte, error) {
    switch op {
    case "isValidPath":
        path := fieldStr(msg, "path")
        ok, err := s.IsValidPath(ctx, path)
        if err != nil { return nil, err }
        return codec.Encode(codec.Record{{Key: []byte("bool"), Value: boolByte(ok)}}), nil

    case "queryPathInfo":
        // ... decode path, call s.QueryPathInfo, encode PathInfo
    case "readDerivation":
        // ... decode path, call s.ReadDerivation, encode Derivation
    case "writeDerivation":
        // ... decode Derivation, call s.WriteDerivation, encode path
    case "computeFSClosure":
        // ... decode path, call s.ComputeFSClosure, encode path list
    case "ensurePath":
        // ... decode path, call s.EnsurePath, encode ok
    case "addTextToStore":
        // ... decode name/content/refs, call s.AddTextToStore, encode path
    case "fetchTree":
        // ... decode attrs map, call s.FetchTree, encode Tree
    default:
        return nil, fmt.Errorf("unknown op: %s", op)
    }
}


func main() {
    if len(os.Args) < 2 {
        fmt.Fprintln(os.Stderr, "usage: eval <nix-expr>")
        os.Exit(1)
    }

    wasmBytes, err := os.ReadFile("libnixexpr.wasm")
    if err != nil { log.Fatal(err) }

    s, err := store.NewLocal()
    if err != nil { log.Fatal(err) }

    ctx := context.Background()
    rt, err := wasm.New(ctx, wasmBytes, s)
    if err != nil { log.Fatal(err) }
    defer rt.Close(ctx)

    result, err := rt.EvalExpr(ctx, os.Args[1])
    if err != nil { log.Fatal(err) }

    fmt.Println(result)
}

Each step produces something testable before moving to the next.

  1. Codec (gonix/codec/) — pure Go, no dependencies. Write encoder, decoder, and round-trip tests for all value types including nested records, variants, lists, and pairs.

  2. WasmStore skeleton — C++ class with all virtuals stubbed to throw Error("not implemented"). Compile natively (not to WASM yet) to verify the class hierarchy, virtual override signatures, and that it links against the existing Store base class correctly.

  3. #ifdef guards — apply minimal POSIX stubs for __EMSCRIPTEN__. Verify the codebase still compiles natively with guards in place (guards must not break the native build).

  4. First WASM compile — attempt em++ of libexpr + libutil + WasmStore without BoehmGC first (-DHAVE_BOEHMGC=0) to get a baseline. Fix link errors from missing symbols iteratively. Expect failures from KJ, capnp, and POSIX headers — these surface the remaining stubs needed.

  5. BoehmGC WASM build — build bdwgc with Emscripten and --spill-pointers as described in §1e. Re-enable -DHAVE_BOEHMGC=1 and link.

  6. wazero host module — implement gonix/wasm/runtime.go and imports.go with a stub store that returns hardcoded valid responses. Verify the WASM module instantiates cleanly and nix_init completes without trapping.

  7. First codec round-trip through WASM — implement isValidPath end to end: WasmStore encodes request, Go host decodes and returns a hardcoded true, WasmStore decodes response. Unit test at the Go boundary.

  8. Full store API — implement remaining import functions one by one, in the order they are exercised by evaluation:

    isValidPath → readDerivation → writeDerivation → addTextToStore
→ queryPathInfo → computeFSClosure → ensurePath → fetchTree

  9. End-to-end smoke test — rt.EvalExpr(ctx, "builtins.nixVersion"). This exercises init, parser, evaluator, and the builtins constant lookup without touching any store operations. Should be the first green test.

  10. Derivation test — evaluate a trivial derivation { name = "x"; ... } expression. Exercises prim_derivationStrict and the full writeDerivation round-trip through the store boundary.