This section provides some notes on how to hack on Lix. To get the latest version of Lix from Forgejo:

$ git clone https://git.lix.systems/lix-project/lix
$ cd lix

The following instructions assume you already have some version of Nix or Lix installed locally, so that you can use it to set up the development environment. If you don't have it installed, follow the installation instructions.

A typical development flow for simple changes in Lix looks like:

If you are using Lix or Nix with the flakes and nix-command experimental features enabled, the following command will build all dependencies and start a shell in which all environment variables are setup for those dependencies to be found:

$ nix develop

That will use the default stdenv for your system. To get a shell with one of the other supported compilation environments, specify its attribute name after a hash (which you may need to quote, depending on your shell):

$ nix develop ".#native-clangStdenvPackages"

For classic Nix, use:

$ nix-shell -A native-clangStdenvPackages

We have a justfile for extra convenient building. It defaults to using ./build as the build directory, and $out (./outputs/out) as the install directory. For most cases, you can clean-build, install, and run the tests with:

$ just setup --wipe && just test

Note

The --wipe argument to meson setup conveniently works whether you have an existing build directory or not.

However, it is mostly, but not exactly equivalent to deleting the build directory first. In particular, previously specified -D build options are preserved with --wipe (for some reason). For example, if you fetch and checkout a new version of Lix, and that new version removes a Meson build option from ./meson.options, and a previous invocation in that build directory explicitly set that option, then meson setup --wipe build will error, complaining about the unknown option. For these cases, just clean will give you a well-and-truly-this-time-for-real clean build.

Because the integration tests require installation to work, just test automatically also calls just install, and Meson helpfully will automatically build any targets that need building when trying to install them.

You can override the build directory or install directory by setting the justfile variables outdir and builddir on the command-line:

$ just builddir=build-before-bisect outdir=out-before-bisect setup
$ just builddir=build-before-bisect test

You'll have to set builddir for every target, but outdir only needs to be set for setup.

You can also run the unit tests and integration tests separately:

$ just setup
$ just test-unit
$ just test-integration

Most justfile targets forward all further arguments to the underlying Meson invocation. For example, to work on both Lix and nix-eval-jobs you can run:

$ just setup -Dnix-eval-jobs=enabled

Note that only targets which don't accept extra arguments can have other targets following them. just clean setup is equivalent to just clean && just setup, but just build test runs the build target with the argument test. This means that if you want to, for example, build with lower parallelism, and then test, you will have to do something like this:

$ just build -j4
$ just test

Finally, the rewrite of the integration test suite, functional2, also has its own justfile target which allows passing extra arguments to pytest. For example, to collect and list all functional2 tests without running them, you can pass pytest's --collect-only argument:

$ just test-functional2 --collect-only

You can also build Lix manually:

$ meson setup ./build "--prefix=$out" $mesonFlags

(A simple meson setup ./build will also build, but will do a different thing, not having the settings from package.nix applied).

$ meson compile -C build
$ meson test -C build --suite=check
$ meson install -C build
$ meson test -C build --suite=installcheck

In both cases, Lix will be installed to $PWD/outputs, the /bin of which is prepended to PATH in the development shells.

If the tests fail and Meson helpfully has no output for why, use the --print-error-logs option to meson test.

If you change a setting in the buildsystem (i.e., any of the meson.build files), most cases will automatically regenerate the Meson configuration just before compiling. Some cases, however, like trying to build a specific target whose name is new to the buildsystem (e.g. meson compile -C build src/libmelt/libmelt.dylib, when libmelt.dylib did not exist as a target the last time the buildsystem was generated), then you can reconfigure using new settings but existing options, and only recompiling stuff affected by the changes:

$ meson setup --reconfigure build

Note that changes to the default values in meson.options or in the default_options : argument to project() are not propagated with --reconfigure.

If you want a totally clean build, you can use:

$ meson setup --wipe build

That will work regardless of if ./build exists or not.

Specific, named targets may be addressed in meson build -C build <target>, with the "target ID", if there is one, which is the first string argument passed to target functions that have one, and unrelated to the variable name, e.g.:

libexpr_dylib = library('nixexpr', …)

can be addressed with:

$ meson compile -C build nixexpr

All targets may be addressed as their output, relative to the build directory, e.g.:

$ meson compile -C build lix/libexpr/liblixexpr.so

But Meson does not consider intermediate files like object files targets. To build a specific object file, use Ninja directly and specify the output file relative to the build directory:

$ ninja -C build lix/libexpr/liblixexpr.so.p/nixexpr.cc.o

To inspect the canonical source of truth on what the state of the buildsystem configuration is, use:

$ meson introspect

The development shell on Linux uses LLD by default for faster link times. This is set using mesonFlags, so to override it, you can simplify re-specify the linker to Meson:

$ just setup -Dc_link_args=-fuse-ld=ld -Dcpp_link_args=-fuse-ld=ld

While using LLD, you may find it helpful to use ThinLTO for even further improvements to link times for incremental builds:

$ just setup -Db_lto=true -Db_lto_mode=thin -Db_thinlto_cache=true

We use Gerrit for all our code review in Lix. Our instance is at https://gerrit.lix.systems.

There's much more information about how to use Gerrit in the wiki section on Gerrit including how to use Jujutsu, how to use the UI and more. The Snix project also has some Gerrit information in their contributing docs.

The gist is that once you have your SSH key and git remote set up, you can send commits for review with:

$ git remote set-url origin ssh://YOURUSERNAME@gerrit.lix.systems:2022/lix
$ git push origin HEAD:refs/for/main

Then, you can request a review via the "Reply" button on the web UI. If you click "Suggest Owners", it will try to suggest the maintainers of the area of the code change to send review requests to. Requesting reviews from multiple people is normal.

We do our best to respond to directly sent reviews in a few days, so feel free to request another reviewer or ask on Matrix if you've not got a response for a while. Keep in mind that Lix is a volunteer project and we have limited bandwidth, so some changes aren't feasible to shepherd through; please check in on Matrix at design time when doing large changes.

Once you get a Code-Review+2 vote on your change, it's rebased on main and CI marks it Verified+1, you're able (and usually expected, so you can have a second chance to check it over) to hit the Submit button to merge it. If the change appears as "Rebase Required", you need to rebase it on main locally or via the Gerrit UI and wait for Verified+1 before the Submit button is made active The Code-Review+2 from before will stick around through trivial rebases so no need to re-request review for a mere rebase.

We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.

However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand. You can log in to the CI via SSO. On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue. If the build still fails on CI issues or all builds are failing this should be reported via Zulip on #T-infra or Matrix on #dev.

To build a release version of Lix for the current operating system and CPU architecture:

$ nix build

You can also build Lix for one of the supported platforms.

Note

You can use native-ccacheStdenvPackages to drastically improve rebuild time. By default, ccache keeps artifacts in ~/.cache/ccache/.

Lix can be built for various platforms, as specified in flake.nix:

In order to build Lix for a different platform than the one you're currently on, you need a way for your current Nix installation to build code for that platform. Common solutions include remote builders and binary format emulation (only supported on NixOS).

Given such a setup, executing the build only requires selecting the respective attribute. For example, to compile for aarch64-linux:

$ nix-build --attr packages.aarch64-linux.default

or for Nix with the flakes and nix-command experimental features enabled:

$ nix build .#packages.aarch64-linux.default

Lix can also be easily cross compiled to the following arbitrarily-chosen system doubles, which can be useful for bootstrapping Lix on new platforms. These are specified in crossSystems in flake.nix; feel free to submit changes to add new ones if they are useful to you.

For example, to cross-compile Lix for armv6l-linux from another Linux, use the following:

$ nix build .#nix-armv6l-linux

It's also possible to cross-compile a tarball of binaries suitable for the Lix installer, for example, for riscv64-linux:

$ nix build .#nix-riscv64-linux.passthru.binaryTarball

It is useful to perform multiple cross and native builds on the same source tree, for example to ensure that better support for one platform doesn't break the build for another. As Lix now uses Meson, out-of-tree builds are supported first class. In the invocation

$ meson setup build

the argument after setup specifies the directory for this build, conventionally simply called "build", but it may be called anything, and you may run meson setup <somedir> for as many different directories as you want. To compile the configuration for a given build directory, pass that build directory to the -C argument of meson compile:

$ meson setup some-custom-build
$ meson compile -C some-custom-build

Lix uses a string with the following format to identify the system type or platform it runs on:

<cpu>-<os>[-<abi>]

It is set when Lix is compiled for the given system, and determined by Meson's host_machine.cpu_family() and host_machine.system() values.

For historic reasons and backward-compatibility, some CPU and OS identifiers are translated from the GNU Autotools naming convention in meson.build as follows:

host_machine.cpu_family() Nix
x86 i686
i686 i686
i686 i686
arm6 arm6l
arm7 arm7l
linux-gnu* linux
linux-musl* linux

Lix can be compiled using multiple environments:

To build with one of those environments, you can use

$ nix build .#nix-ccacheStdenv

for flake-enabled Nix, or

$ nix-build --attr nix-ccacheStdenv

for classic Nix.

You can use any of the other supported environments in place of nix-ccacheStdenv.

The clangd LSP server is installed by default in each development shell. See supported compilation environments and instructions how to set up a shell with flakes or in classic Nix.

Clangd requires a compilation database, which Meson generates by default. After running meson setup, there will already be a compile_commands.json file in the build directory. Some editor configurations may prefer that file to be in the root directory, which you can accomplish with a simple:

$ ln -sf ./build/compile_commands.json ./compile_commands.json

Configure your editor to use the clangd from the shell, either by running it inside the development shell, or by using nix-direnv and the appropriate editor plugin.

Note

For some editors (e.g. Visual Studio Code), you may need to install a special extension for the editor to interact with clangd. Some other editors (e.g. Emacs, Vim) need a plugin to support LSP servers in general (e.g. lsp-mode for Emacs and vim-lsp for vim). Editor-specific setup is typically opinionated, so we will not cover it here in more detail.

To build the manual incrementally, run:

meson compile -C build manual

mdbook-linkcheck does not implement checking URI fragments yet.

The built manual is in build/doc/manual/manual/index.html.

The build checks for broken internal links. This happens late in the process, so nix build is not suitable for iterating and it's recommended to use the meson command above instead.

@\docroot\@ provides a base path for links that occur in reusable snippets or other documentation that doesn't have a base path of its own.

If a broken link occurs in a snippet that was inserted into multiple generated files in different directories, use @\docroot\@ to reference the doc/manual/src directory.

If the @\docroot\@ literal appears in an error message from the mdbook-linkcheck tool, the @\docroot\@ replacement needs to be applied to the generated source file that mentions it. See existing @\docroot\@ logic in doc/manual/substitute.py. Regular markdown files used for the manual have a base path of their own and they can use relative paths instead of @\docroot\@.

Doxygen API documentation will be available online in the future (tracking issue). You can also build and view it yourself:

# nix build .#hydraJobs.internal-api-docs
# xdg-open ./result/share/doc/nix/internal-api/html/index.html

or inside a nix develop shell by running:

$ meson configure build -Dinternal-api-docs=enabled
$ meson compile -C build internal-api-docs
$ xdg-open ./outputs/doc/share/doc/nix/internal-api/html/index.html

A coverage analysis report will be available online in the future (FIXME(lix-hydra)). You can build it yourself:

# nix build .#hydraJobs.coverage
# xdg-open ./result/coverage/index.html

Metrics about the change in line/function coverage over time will be available in the future (FIXME(lix-hydra)).

doc/manual/rl-next contains release notes entries for all unreleased changes.

User-visible changes should come with a release note. Developer-facing changes should have a release note in the Development category if they are significant and if developers should know about them.

Here's what a complete entry looks like. The file name is not incorporated in the final document, and is generally a super brief summary of the change synopsis.

---
synopsis: Basically a title
# 1234 or gh#1234 will refer to CppNix GitHub, fj#1234 will refer to a Lix forgejo issue.
issues: [1234, fj#1234]
# Use this *only* if there is a CppNix pull request associated with this change.
prs: 1238
# List of Lix Gerrit changelist numbers.
# If there is an associated Lix GitHub PR, just put in the Gerrit CL number.
cls: [123]
# Heading that this release note will appear under.
category: Breaking Changes
# Add a credit mention in the bottom of the release note.
# your-name is used as a key into doc/manual/change-authors.yml for metadata
credits: [your-name]
---

Here's one or more paragraphs that describe the change.

- It's markdown
- Add references to the manual using @docroot@

Significant changes should add the following header, which moves them to the top.

significance: significant

The following categories of release notes are supported (see maintainers/build-release-notes.py):

The credits field, if present, gives credit to the author of the patch in the release notes with a message like "Many thanks to (your-name) for this" and linking to GitHub or Forgejo profiles if listed.

If you are forward-porting a change from CppNix, please credit the original author, and optionally credit yourself. When adding credits metadata for people external to the project and deciding whether to put in a display_name, consider what they are generally known as in the community; even if you know their full name (e.g. from their GitHub profile), we suggest only adding it as a display name if that is what they go by in the community. There are multiple reasons we follow this practice, but it boils down to privacy and consent: we would rather not capture full names that are not widely used in the community without the consent of the parties involved, even if they are publicly available. As of this writing, the entries with full names as display_name are either members of the CppNix team or people who added them themselves.

The names specified in credits are used as keys to look up the authorship info in doc/manual/change-authors.yml. The only mandatory part is that every key appearing in credits has an entry present in change-authors.yml. All of the following properties are optional; you can specify {} as the metadata if you want a simple non-hyperlinked mention. The following properties are supported:

Releases have a precomputed rl-MAJOR.MINOR.md, and no rl-next.md. Development releases have a generated rl-next.md.

Experimental and deprecated features, global settings, and builtins are generally referenced both in the code and in the documentation. To prevent duplication or divergence, they are defined in data files, and a script generates the necessary glue. The data file format is similar to the release notes: it consists of a YAML metadata header, followed by the documentation in Markdown format.

Experimental and deprecated features support the following metadata properties:

Experimental feature data files should live in lix/libutil/experimental-features, and deprecated features in lix/libutil/deprecated-features. They must be listed in the experimental_feature_definitions or deprecated_feature_definitions lists in lix/libutil/meson.build respectively to be considered by the build system.

Global settings support the following metadata properties:

Settings are not collected in a single place in the source tree, so an appropriate place needs to be found for the setting to live. Look for related setting definition files under second-level subdirectories of lix whose name includes settings. Then add the new file there, and don't forget to register it in the appropriate meson.build file.

The following metadata properties are supported for builtin functions:

New builtin function definition files must be added to lix/libexpr/builtins and registered in the builtin_definitions list in lix/libexpr/meson.build.

The following metadata properties are supported for builtin constants:

New builtin constant definition files must be added to lix/libexpr/builtin-constants and registered in the builtin_constant_definitions list in lix/libexpr/meson.build.