(1) I get mired in packaging low level transitive dependencies and things which seem like they should be easy end up being nearly impossible
(2) Basic things like figuring out the argument types for a particular nixpkgs function take an insanely long time. It always involves grepping around the whole repo (nixpkgs is huge for those who don't know) for invocations to find the variable that gets passed into that function and then grepping around the repo for the function that produces that variable and so on until you get to the source type. Things have awful names and documentation is sparse.
I'm a big believer in Nix as an abstract concept, but the execution has been a miserable experience for me to the extent that I can't make sense of people who report such positive experiences. I would think I'm doing something wrong, but so many people have corroborated my experiences and when I bring specific issues to experienced Nix users they also get stuck.
Ultimately, I've reluctantly gone back to dealing with Docker and system package managers because their failure modes are rarer, more predictable, and more easily worked around.
EDIT: and I haven’t touched on the problems associated with using Nix in an organization. It seems like invariably one or two people become “the Nix experts” and they become the bottleneck for the whole organization, every team needs to coordinate virtually every interesting activity with them, etc.
(1) I do find that it's very rare having to implement a low level transitive dependency from scratch (most low-level system dependencies are already there, and for dependencies of a specific programming-language ecosystem you'd usually auto-generate them from things like `yarn2nix` or `poetry2nix`). However, I agree that once you need to do that for non-trivial library it quickly becomes painful and require intimate knowledge of both the build system and Nix environment.
(2) This is my main issue. My workflow is also having a local `nixpkgs` clone, and go find the function and the parameters I can use every time I don't remember how to use it. I also think that it is sustainable. There are many exciting improvements going on with the new CLI, so hopefully we will get nicer error messages and better tooling support in near future.
> but the execution has been a miserable experience for me to the extent that I can't make sense of people who report such positive experiences.
There are pain points, but the advantages for me were indispensable. Being able to pin dependencies accurately, reproduce an exact development environment deterministically, and use the single language to define applications, configurations, and even entire system images is a great benefit that I would need to use 10+ tools with individual quirks otherwise.
Nix certainly has a steep curve, but simple things are not that hard. I got committed to migrate to Nix one Monday. I spent the whole morning reading about it. In the afternoon, my workstation was already up and running. On the same evening, I packaged two exotic things I need which weren't on NixPkgs. Next day, I fixed my favorite window manager, which was broken on NixPkgs. I maintain all these on NixPkgs now.
Complicated things are not that easy, and I would not have been able to do all this if I had encountered non-trivial issues in the process. I also totally understand that the learning curve is steep. Nix has too much legacy stuff and cruft built in. There's the classic Nix command, Nix 2.0 and Nix flakes. They are all coexisting and not very well documented. Nix needs more manpower, funding and tooling.
With that said, declarative stuff is poised to be harder for non-trivial stuff. Just like Haskell is harder than C, NixOS is harder than ArchLinux or Alpine. I still very much find it worth the effort for simple workflows. I can now update remote machines without fearing breakdowns and state is very explicit.
I mean... it's great that you did all that, but you do realize that going that far makes you a power user, right?
Most users are going to give up if they can't figure out how to install their window manager or other essential software (eg installing Docker or Steam is ridiculously hard if you don't know that they have special hard-coded config values; it's almost like a game you can only beat if you find the cheat codes).
> With that said, declarative stuff is poised to be harder for non-trivial stuff. Just like Haskell is harder than C, NixOS is harder than ArchLinux or Alpine. I still very much find it worth the effort for simple workflows. I can now update remote machines without fearing breakdowns and state is very explicit.
Hard disagree. I could totally see a world where NixOS is as easy to install as any other distribution. But the developers are going to have to put a lot of work into smoothing out wrinkles before it gets there.
The way I see it, it's the foundational ideas that are important—but there are a few key principles beyond the one covered in the blog post.
Separately, today, I think the foundational advantages Nix has more than justify the learning curve—but that's a less obvious idea and I understand why it's hard to convince other people about it :).
I hope that'll change over time, because yeah, the foundations are /so/ powerful, even today.
It's hard to find out how to do things, and often you find two examples with a partial solution that each use a very different incompatible way to get there.
I think the core idea of Nix is good, but we need a decent programming language designer to have a go at a new language for packaging.
Hard disagree on the "decent programming language designer", though. Nix-the-language is beautiful. It's tiny, with a minimum of concepts to learn, and very simple, regular syntax. At the same time, it's not so spare as say, Lisp or Scheme. It's really good. The implementation of that language could use some work - interpreter speed and debugging tools come to mind rather quickly.
I think the problem you've encountered is nixpkgs. It's a huge, sprawling code base with a lot of legacy code and a number of paradigms that were enthusiastically adopted and then later supplanted. In contrast to the language and tools which are well documented, there's basically no documentation, and what little there is hasn't been kept up-to-date as things change.
Still though, I think the biggest issue facing nix users is that the rest of the computing world works on the assumption of mutable software installations. The way pip assumes that the python interpreter and the python packages that have been installed are locked in a death grip drives me nuts. Even if you don't go full nix-store, could we at least separate those things? Even node gets that right.
On this I definitely agree. Like I said I do think Nix is the right thing to do, I'm just thinking the how could be better.
this is most painful, because a lot of packages do tons of tricks that make it very difficult to get reproducibility. Like downloading dependencies at build time, non standard places of installing things hardcoded paths etc.
If you are tried cross compile arbitrary packages it might be familiar. A lot of projects don't know how to do things correctly, or doesn't care and just want their code to work.
To that, other environments have their own quirks. For example the way most nodejs applications work they do expect their dependencies in current working or parent directory.
All those things require some patching. If you can get the patch upstream that's awesome, otherwise you'll have to maintain it.
What is great about other distros, is that if you try to use some exotic library, you can just try things randomly until you get it compiled and installed in /usr/local :)
Imagine if you had RedHat installed, and it would impose that you can't run anything unless it's packaged in rpm. If you would have to prepare a .spec file and run rpmbuild command. I bet experience wouldn't be far off.
Unlike RedHat Nix does force this and this is a double edged sword. It creates this pain, but on the other hand it forces some kind of order.
BTW: if you want it hard enough you can still install packages the traditional way (but likely would have to specify --prefix to somewhere inside /home than /usr/local)
1. The language is a significant problem. If you get everything right, it will do what you want. That's the only nice thing I can say about it. Almost every error it will give you will be misleading. I haven't had such a hard time figuring out errors since early C++ template metaprogramming. And frankly, that was easier.
It needs real diagnostics, and that frankly, that means it needs real types. But that won't happen. Too much of the informal type system of nix is "whatever keys that library function uses, go look in the source."
2. Half-implemented packages. Try walking out of the golden path on a package, without having to modify its nixpkgs entry.
Between the two, it's more nerd-bait than usable platform.
1. https://github.com/tweag/nickel - it is a typed language, that might in the future replace nix language
2. looks like likely is addressed by flakes
BTW: the Nickel has chance to end up replace Nix language as I saw author of Nix (Eelco) is also contributing to it. I don't think it can transpile to a nix code.
I don't disagree with you, but one of my skepticisms about this is that Docker and Dockerfiles made the same promise like eight years ago, and at least as a small time operator (eg, homelab), it never really felt like the system was as stateless as it was telling me it was. There were always logs, indexes, caches, database content, and all the rest of it, and all of this had to be carefully volumed-out with those data containers being protected and versioned and carried forward independent of the throwaway Dockerfile-defined "application" containers.
So far my experience of NixOS is that it doesn't particularly offer a new solution here, but at least it doesn't pretend anything. The configuration.nix "defines" the whole system, but obviously doesn't touch anything in /home or /var, so that's on you to back up, migrate, whatever.
https://github.com/nix-community/home-manager
> or /var, so that's on you to back up, migrate, whatever.
Are they using Nix wrong? Or is this seriously a limitation of the current state of the tooling?
If it's a limitation of the tooling, that almost sounds like satire. I.e., someone builds a powerful functional DSL, uses it to painstakingly construct a monstrous package graph, then gets tired and wonders, "Wouldn't it be great if there were a language and data structure that were both custom suited to automating the grep'ing of argument types for my DSL's package graph..."
Of course, sometimes those kinds of changes just never get merged, for a variety of well-understood open-sourcey reasons, eg: https://github.com/pocoproject/poco/pull/3105
But in short, the maintainers are mostly uncompensated, and therefore have no sense of urgency or obligation to a faceless stranger making a drive-by contribution. Every accepted contribution is something that has to be understood, supported, and even justified by the project's maintainers going forward, so each one is an opportunity to take on more work and risk, and obviously by definition almost all them are things the maintainers didn't themselves "need" or they would have done it themselves. So each new branch or bugfix is relevant to a codepath that no one involved in the planning, testing, or release-preparation for that project has a first-class stake in. Each new capability added is an opportunity for there to be shrieks following a release because it was inadvertently broken in the course of some other bit of development.
And, as a practical benefit, I found setting up borg backups for /home on NixOS a lot easier than I expected :)
5. It's essentially a source distribution with caching. If you want to add say a kernel patch, or any other type of patch, it's straightforward. Whereas running Debian and finding the solution to some problem is building a modified package, my usual conclusion was just to bear the problem until it was fixed upstream, rather than suffer the maintenance burden.
6. The top-level config is front and center, and stateless. No messing about with update-initramfs, wondering what the heck the initrd is actually doing etc.
7. Taming the complexity of system software's configuration state. For example, there is no mucking with the systemd mess of /etc/systemd vs /lib/systemd and its assorted symlinks, as this configuration is handled at a higher level.
And for Nix I'd add
8. Dead simple cross (or emulated) compilation toolchains. Building A Cross Compiler used to be a Big Deal, with Nix it's basically like any other declaration to build an entire cross/emulated environment.
But building packages for Debian is dead simple, it is literally one command. Adding a patch to an existing package, and then rebuilding it, is a common operation.
Can it be that nothing is happening in this regard or is that really so that people only list same cons over and over again? :)
Edit: after searching about it, looks like this was the same issue with /nix in the root, and disabling SIP was one solution at the time. Currently the solution appears to be creating a separate volume and using /etc/fstab and /etc/synthetic.conf to set it up[1]. The installation script does it automatically.
https://nixos.org/manual/nix/stable/installation/installing-...
Does that mean we get downloadable precompiled binary packages too? If not, why not?
The other bonus is that it takes very little effort to share your own binary cache—either privately (so that you or your company only need to build something once) or publicly (so that people installing your open source tool don't have to build from source). There's even a really well-designed commercial service called [Cachix][1] that provides a totally turn-key binary cache, including a free tier for open source projects.
The nice thing is that it sped up building by 50% and also extra bonus with things is that with merges, if files didn't change nix will finish building instantaneously.
This is similar with nix-shell, where for example with python you enter "virtualenv" like environment almost instantly without downloading dependencies over and over. Yes, you could get close to do that with CI by using some caching, but this is out of the box, due to nix understanding dependencies.
Also another great thing is that so far, I never needed to purge cache, which often happens with normal CI caching.
1. Nix store: what the article covers: multiple versions of the same package, "virtualenv for everything"... etc
2. Reproducibility: because package definitions are self-contained, I can pin and reliably reproduce everything for my packages. And I do mean everything.
3. Flexibility: Nix works for system packages on NixOS as well as user packages on any Linux or macOS. I've been sharing like 70% of my system config between Linux and macOS, and I don't know of any other tool that can do that.
4. Programmability: package definitions and config are extensible and composable. Yes, learning an ad hoc language to do this is a pain, but once I learned it I got a higher-level and better-abstracted way to define and combine packages than any other tool I've used. It's like going from C to OCaml. At this point I've started hating Dockerfiles because they can't easily be composed and don't provide any real facilities for abstraction.
Ultimately, Nix took system management from feeling like an operational activity—maintaining and changing a bunch of mutable state to run the system—to declarative programming. Learning curve aside, making system management a reproducible programming activity with no system state to worry about has been a massive improvement.