Nix distributed builds

Building nix derivations can take some time and resources. When I run nixos-rebuild switch --upgrade and it happens that I need to build a kernel or a package from source, my notebook gets very busy. My server has much more resources and significantly more RAM than my Dell XPS. Turns out I can use this workhorse and shift some workloads.

Setup your remote server

Using the distributed builds feature of Nix is not that complicated.

• First, you need to install Nix on the remote server.
• Then you can test whether your local machine can ping the remote Nix store: nix store ping --store ssh://workhorse

Troubleshooting

You might run in this error message:

bash: nix-store: command not found
error: cannot connect to 'workhorse'

This means that you need to check on your remote server whether all necessary PATH variables include the path to your Nix binaries. The nix store command uses a non-interactive shell which usually has a limited set of paths in its PATH. You can check the contents of your non-interactive PATH on the remote host using these commands on your local host:

ssh workhorse echo \$PATH
ssh workhorse sudo echo \$PATH

Both should include the path to your Nix binaries (e.g. /root/.nix-profile/bin). If that’s not the case, you need to edit one of those files:

• .zshenv (if you are using zsh as your default login shell on the remote host)
  • Add this line → export PATH="/root/.nix-profile/bin:$PATH"
• .bashrc (if you are using bash as your default login shell on the remote host)
  • Make sure to add this line → export PATH="/root/.nix-profile/bin:$PATH" on top of this line → [ -z "$PS1" ] && return. Everything after this is only loaded for interactive shells.

Since nix-store is sometimes called with sudo, the path to the nix-binaries should also be added to the secure_path. Just type visudo and add a /root/.nix-profile/bin to the secure_path line.

Usage

Just add --build-host root@workhorse to your nixos-rebuild command.

sudo nixos-rebuild switch --upgrade --build-host root@workhorse

Further links

• NixOS manual: advanced topics - distributed builds
• Stackoverflow: secure_path