It was long overdue, but it finally happened. I migrated most of my stuff to nix. While it wasn’t a game changer for my day to day computer experience, it made self hosting a lot more enjoyable for me. For the uninitiated, nix is a functional programming language that allows you to build and package software in a declarative and reproducible way.
The point of this post is not to go any deeper into what nix is or how to use it. There are many articles that already do that. Instead, I want to write about my personal experience adopting nix and some of my thoughts on it.
The migration was done progressively, so I could back out any time if needed. I started with simple things like dotfiles, continued with packaging my side projects, and ended up migrating my personal VPS to NixOS.
The first thing I migrated was my dotfiles and user specific programs1 by using a standalone home manager setup to manage them.
The reason I went with a standalone home manager setup was to make my setup system agnostic. I have a work MacBook, a personal MacBook, and a personal Linux box, and I wanted my setup to work on both macOS and Linux without needing to have separate nix-darwin and NixOS setups.
The standalone home manager setup also has the benefit of not needing sudo for changing your configuration with
home-manager switch, compared to installing it as a NixOS/nix-darwin module. This is especially important for my work
machine where I need to justify sudo permissions. It’s easy to justify it when installing nix, but it would be quite
tedious to justify it every time I want to change my dotfiles.
After migrating my dotfiles, my next goal was to migrate development environments and build scripts for some of my more active side projects. This was a very straightforward task. I used nix flakes together with direnv to package my side projects with nix.
Part of packaging the side projects with nix was also replacing GitHub Actions with garnix.
This was hands down the most pleasant CI experience I’ve had in my life. Once the project is packaged with a nix flake, all
that’s left is removing the ugly yaml config.
I have a VPS on which I host the website you are currently reading, together with some of my side projects. Previously, this was a Debian machine. After migrating my dotfiles and packaging my side projects as nix flakes, I decided to try moving the whole server to a NixOS installation.
After I got the basic configuration (disk, networking, users, …) for the NixOS installation dialed in, moving the projects was very easy. Since they were packaged as nix flakes from the previous step, I could just include the project as the server’s flake input and call it from a systemd service (or whatever else I wanted).
One very nice feature of NixOS that I use is the --target-host flag of the nixos-rebuild switch command.
This allows me to build the whole system (together with all my side projects) on my beefy desktop machine, and then just
deploy it to the remote server.
Overall I am very pleased with the results of moving my stuff to nix. However, like everything in life, it’s not perfect. I am going to share some thoughts on the good and the bad in no particular order.
An obvious good feature is reproducibility. It is absolutely amazing that I can write a nix derivation to build (and check) my project, and the thing will work on every machine without me worrying about dependencies, versions, and other details.
The reproducibility is also the basis for other cool features, like garnix CI. It lets you share build artifacts,
including intermediate ones, between machines. This allows for a shared cache (like cachix).
For instance, I can build, run tests, and run other checks on my beefy machine. Then I can upload all the artifacts to
a cache shared between my machine and garnix CI. When garnix runs nix build (or similar), nix will see that the
requested derivation is already present in the cache and just go “Done!”. This means pipelines are crazy fast.
It also makes server NixOS deployments magical. You can build NixOS on a machine different from the one you’ll
deploy the config to. In fact, the build machine and target machine can be different from the machine where you are invoking
the nixos-rebuild command. This means I can run nixos-rebuild on my MacBook from the comfort of the couch,
the build happens on my beefy Linux desktop, and finally it’s deployed to a VPS on the other side of the continent.
How cool is that!
A final great thing about nix that I want to point out is an even more obvious fact: nix is a programming
language. For managing my dotfiles, this means that I have access to things like an if statement. When managing
a work Mac, personal Mac and personal Linux, this is really helpful.
Now let’s focus our attention to some of the uglier parts of nix. The ugliest and most obvious wart is definitely the docs. Getting started with nix is very daunting mostly because docs are all over the place, at times outdated, and at times lacking explanation. This is absolutely the one thing that nix needs to improve on the most.
Another problem you’ll run into quickly is disk usage. Nix tends to explode pretty quickly, and you need to set up garbage
collection to not run out of disk space. Not that garbage collection, let alone automatic garbage collection, is explained well in the docs…
A bit less obvious problem, but still a problem, is the amount of inodes nix consumes. I learned the hard way that
nix should probably not be used on an ext4 file system, since ext4 has a limited number of inodes set at the time of
formatting. When you run out, you’re out. I followed an outdated tutorial on installing NixOS (on the official wiki…),
and installed NixOS on my VPS using ext4. After a couple of days I had to reformat my disk to btrfs and reinstall
the OS. Thankfully I was using nix and just had to do nixos-rebuild switch with my backed up config :).
The last problem I want to point out is just the general design of the language. I would really love it to have a better type system that would catch more errors sooner. Instead, you have to build the derivations to get the errors, and a lot of the time error messages are quite bad and make it difficult to understand what the actual underlying error is.
At the end of the day nix is the best we’ve got when it comes to building software and configuring reproducible systems, even though it’s not perfect. I would recommend anyone to try it because it does have interesting ideas and approaches. But don’t expect it to improve your computer experience. If you manage just one or two computers nix is fun to tinker with, but doesn’t improve your life in a meaningful way.
However, if you are into self-hosting, NixOS is a must in my mind. I would not go back to hosting my own server
without it running NixOS. The fact that I can see exactly what is running on my server and where, which ports are
used, which users are on the system, where the secrets are, where and what the configs are, and more, is absolutely
stellar. Combine that with the ability to roll back to a previous config any time you want, and you get an absolutely amazing
way of managing servers. And if that is not enough, the cherry on top is the ability to experiment with
new server configs on a VM, and when I am happy with it I can just do nixos-rebuild, with my VPS as the target instead of the VM.
So to recap: try it out. If you have only one or two computers don’t expect any sort of revelation, but if you are into self-hosting, it’s amazing and you should use it.