Like you, I find Nix hard to use and the docs confusing (beyond trivial examples). The amount of time the Nix mafia spends proselyting could be better spent making better docs
But it's no wonder that people keep commenting that Nix is an awesome idea that solves a lot of problems. That's because it's an awesome idea that solves a lot of problems (yeah, like Rust).
It is also completely undiscoverable (not completely unlike Rust), and that's why almost nobody uses it. But that's a matter of improving the documentation or maybe fixing one or two superficial problems on the language.
Based on a quick check of their Wikipedia page, Nix has been around for 15 years, since 2007.
So those for sure look like structural problems. I don't know which ones exactly, but paraphrasing Tolstoi, popular software products are popular in much the same way, while unpopular ones are unpopular in their own unique ways.
In my experience almost no open source project that hasn't entered the mainstream in its first 10 years manages to turn the ship around, unless it lucks into a major change of environment. Programming languages are sometimes exempt if they have a "killer" library or framework pop up.
That said, I haven't found Nix hard to use, quite the opposite. The new 'nix' command is pretty self explanatory, it's just a little incomplete in spots. And the Nix language is quite simple and easy to understand if you have ever touched a functional language. Compared to all the other Linux stuff I have played with over the years, it was a very pleasant experience so far.
HN tends to see success in terms of adoption rate & ranking. Nix is on another plane. It's beyond critical mass. And for people like me, it is a huge force multiplier. But I had to grind my way to the point where that is true.
I tend to not bother selling people on it. People who want what Nix does will find it at some point. So I just focus on using Nix myself in a world of worse tools. Feels great.
Evidence of that is that you can just talk about it by name here on HN without explaining what it is.
Is Haskell mainstream? What about committing murder?
And yes, this is 100% Nix's fault
I would say that HN is far from mainstream.
The reality is that the docs are fine. The reference material is comprehensive, and there are some very good tutorials and introductory essays to get people started. The problem is that it's difficult. Nix is very, very different from what people already know, and it takes quite a bit of effort to learn how it works. Most people just aren't prepared to do that much work just to install some software.
It helps that people who are into Nix most often use NixOS. NixOS is the only distro I've written packages for. It's not even particularly hard since there's a lot of inspiration to find in the official and community repos + people sharing personal configurations.
The UX wasn't the best but it's improving.
It remains extremely reliable which is not something that you could say of other systems.
Docker is more pragmatic unless you spend hundreds of hours on Nix, in my opinion.
Most pilot text books are absolutely horribly written, finding a good one takes significant effort. Reading just an ok book shines a light on how absolutely trash the majority of pilot text books are and that it is absolutely bad "documentation" that is partially to blame for it being difficult to learn.
Just like inherent complexity in some software or business problems.
Maybe airplanes wasn't the best thing to compare with -- the comment below compares with C instead, maybe that's better. I never flew a plane.
then show me where `nix-env`'s `--arg` and `--argstr` is documented.
--arg name value
This option is accepted by nix-env, nix-instantiate and nix-build. When
evaluating Nix expressions, the expression evaluator will automatically try to
call functions that it encounters. [...]https://nixos.org/manual/nix/stable/command-ref/opt-common.h...
Oh I think you're using HN wrong. To learn about the subject you can read the actual article that's linked to. No guarantees about the comments section.
I've never heard of asdf and popped in here to read about it, people's experiences with it, etc. If I want to read about Nix/OS I'll find a thread on that, or go to /r/nixos, etc.
And what are you doing :-)
But, since you're downloading & running code written by someone else, it seems weird to talk about that but ignore "what if this program I'm downloading does something malicious".
[1] https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b...
Nix provides a level of flexibility that other solutions simply do not offer. If a package in Nix doesn't fit your requirements for whatever reasons, you can create a modified version of a package with ease. For example, say that you need a version of Nginx built against a custom version of OpenSSL. You can do just that with a few lines of code:
let
mypatch = pkgs.fetchpatch {
url = "https://example.com/bugfix-for-openssl.patch";
sha256 = "...";
};
openssl = pkgs.openssl.overrideAttrs (old: {
# add build flags
configureFlags = old.configureFlags ++ [ "--enable-foo" ];
# add dependencies
buildInputs = old.buildInputs ++ [ pkgs.foo ];
# add patches
patches = old.patches ++ [ mypatch ];
});
in
nginx.override { inherit openssl; }
Nix will even know which packages it'd need to build locally instead of downloading a prebuilt binary.You can't do that with other common package managers. You're stuck with whatever the package manager provides you. So if you're not in the happy path, you're out of luck. You'd either have to give up or build from scratch. If you need a bugfix for a particular package, you'd have wait until the fix reaches the package repository. When I was using Ubuntu, that was often until the next major release. None of this stuff is a problem with Nix, which allows for customization with very little effort.
It's easy to learn about Nix if you know where to look for. Nix Pills [1] would be a good start. The core language is the easiest part. It's JSON, but with functions and variables for proper abstraction. The documentation, while imperfect, is quite extensive compared to a majority of other high profile open source projects. If the documentation fails you, the Nixpkgs repository [2] is an even more rich source of information. The code is well organized, and I was able to get familiar with writing Nix packages fairly quickly by grepping the codebase. And finally, Nix has a sizable community so you could always ask if you're stuck.
I understand you are saying it with the best intentions but it might sound like not the most correct way of defending a tool being criticized for its lack of documentation or easy of use.
Ultimately documentation doesn’t solve usability problems it just makes note of them.
Here's the simplest example of an utter failure when I tried nix: installing ruby 2.2. It's very possible I did something totally wrong, but when I have to use ruby 2.2 and nix cannot do it, it sort of kills your awesome example of being able to override openssl.
Another extremely frustrating experience was the bifurcation between nix and nixos. Clearly one experience is preferred over the other. So many blogs and docs talk about things like HomeManager and other configurations that are nixos specific.
On top of nixos/nix, there's also shell.nix/flakes. There's just too much development in nix right now to make sense of what's going on.
Considering that ruby 2.2 was released in 2014, I can't say I'm surprised. It's too much effort to continue maintaining something so old. But if you really want to use it, you could try using the package definition from an older commit of Nixpkgs:
let
nixpkgs1709 = pkgs.fetchFromGitHub {
owner = "NixOS";
repo = "nixpkgs";
rev = "e09c0adc63d10249dac8f90313f91e1050861d3c";
sha256 = "sha256-Di9D0gvaESV3JmX/kW2uEJ68QDlAke23t19bImTXsJ8=";
};
pinnedPkgs = import nixpkgs1709 { };
in
pinnedPkgs.ruby_2_2
> Another extremely frustrating experience was the bifurcation between nix and nixos.I'm not sure what you mean. Nix is the package manager while NixOS is an entire distro based on Nix the package manager and its configuration language. They have clear separation of concerns. Keeping it that way has actual benefits too, namely portability. It allows you to use Nix on a wide range of platforms including non-NixOS Linux distros, macOS, and even BSDs to some extent.
> On top of nixos/nix, there's also shell.nix/flakes.
Flakes are currently an experimental feature and not meant for wide use yet. So if you're just getting started with Nix, I'd recommend looking into it later until you're sufficiently comfortable with Nix.
They're not fundamentally different, though. Flakes formalize the conventions for writing Nix expressions to make them more reusable.
I've been using nix as my daily OS for six months and I'm able to be productive. At the same time, I'm often confused as to whether I should use nix-shell, nix-env or modify my configuration files.
Thanks for the great comment.
Also rollbacks in nixos are amazing.
On the other hand newly packaging something can be really daunting.
That's less of a guarantee with a collection of shell scripts.
Nix provides better abstractions and composability to make that experience far less painful. Note that in my previous example, I didn't need to specify how to build OpenSSL or Nginx. I was able to reuse the definition from Nixpkgs and apply my own customizations to it.
I also feel like nix is immediately tangible and beneficial in a way that makes us want to get to the power user level, but I don't feel like wanting to learn, say, kubernetes means one automatically wants to learn to be a cluster operator. The scope and complexity are great, and I have experienced all of the things you have described as well. Now that I am fairly confident in my abilities, I think it's important to make sure we find ways to help everyone get the most out of it, even if it doesn't involve scaling the wall that is the learning curve.
Can you clarity what you mean?
What do you mean by "trap"? Which (if any) of the following concepts factor into your understanding? Broken expectations? Premature optimization? Over-engineering? Lock-in?
Are you suggesting that many users of the Nix package manager "fall into a trap" and begin using NixOS?
What do surveys and/or data show about usage of just the Nix package manager? Combined with NixOS?
The Nix ecosystem is attractive to me at a high level because it looks like it has the possibility to revolutionize a lot of the problems that come up with an increased focus on devops culture. The problem with the Nix ecosystem itself is that it is larger and more varied than it might appear.
There are really three large entities in the ecosystem: nix the language, nix the package manager (nixpkgs), and nix the OS (NixOS). Despite the fact that there is a lot of overlap in some of the details of these three things, they are easily understood as separate entities. There are a bunch of smaller secondary entities such as nix-darwin, home-manager, devshell, and digga (formerly devos). The aim of these entities is usually to supplement, enhance, replace, or reproduce features of nixpkgs and/or NixOS.
Each of these entities has its own learning curve. It takes surprisingly little nix knowledge to use it as a dev environment generator, but nixpkgs knowledge will help a bit more. Even within nixpkgs itself, each major language has a large ecosystem built up around facilitating package building and deployment, and these are extremely heterogeneous. Some of them require that you regenerate the entire matrix of package dependencies for the package manager (javascript, haskell). Some of them have tooling that allow you to get started with almost no effort (mach-nix for python). These all have their own peculiarities. NixOS has a ton of modules, all rather heterogeneous, on top of requiring more careful knowledge of how to put together a system than most other distros. And nix the language without nixpkgs is very small, and some of what it does have built in is necessary so it can bootstrap an actual stdlib from nixpkgs.
The trap is sprung once you've been sold on the idea, and even seen it shine through some things, but feel like you have an entire wall to climb to get to a point of general competency. All of these things I've described, and even some of their subgroups, require new knowledge for a more complete understanding. I do not understand the details of many of the things I have described, and I would consider myself a power user at this point. What looks like one gigantic, insurmountable cliff is really dozens of much smaller cliffs, many of which are not necessary, at least not to most people.
At one point I wanted to test my skills in a really generic way, so I helped fix broken packages on macOS during one of the two yearly regression periods where the community tries to get the CI builds passing. I picked the packages completely at random. Most of what I ended up fixing was C/C++ based, and I don't normally work in it, or I haven't for many years, but I feel like I learned a good bit about cmake. I don't really write Haskell, but I was able to fix a bug in a solving library having to do with a missing portability shim. I didn't have help for any of this, and in some cases I was either too stuck to fix the build or unable to fix it due to problems beyond my control.
All of this is to say that I think getting to this point is a lot less painful if one takes a use case, preferably a narrow one, and focuses on learning whatever is necessary in the service of that use case. Some good topics are dev environments, using nix as an app/project builder, using nix to build containers, using nix to manage remote machines (the difference between these and personal machines is that the scope of these machines tends to be much smaller). I think people try to take on too many of these things simultaneously and end up overwhelmed. That's not to say that NixOS isn't worth learning, but I think many people could realize most or all of the real-world benefit if they use nix on an OS they're comfortable with. Coming back to my comparison of being dropped in the Romanian countryside, if you want to learn Romanian, start there. If you want to experience the countryside, maybe a guided tour is the way to go. If you want to learn about the life of Ligeti, you might not even need to learn Romanian! But total immersion is difficult and draining, and a failed immersion experience doesn't mean Romania is a bad place. So it is with Nix.
package manager (whether on Darwin or any linux distro), to have some tools globally available:
nix-env -iA nixpkgs.${some-package}
per project, most shell.nix can just look like this: {
pkgs ? import <nixpkgs> {},
}:
let
some_var = some_value;
in pkgs.mkShell {
buildInputs = [
pkgs.some_package
...
];
shellHook = ''
# this is bash, so, whatever floats your boat
'''
e.g asdf, only much more generic with full non-leaking package management: {
pkgs ? import <nixpkgs> {},
}:
let
some_package = pkgs.some_package_1_2;
in pkgs.mkShell {
buildInputs = [
some_package
pkgs.some_other_package
...
];
shellHook = ''
export SOME_VAR="some_value"
...
'';
}
ruby/rvm/rbenv/bundle exec (example for rails >= 6): {
pkgs ? import <nixpkgs> {},
}:
let
ruby = pkgs.ruby_2_7;
python = pkgs.python27;
node = pkgs.nodejs-14_x;
in pkgs.mkShell {
buildInputs = [
ruby
pkgs.sqlite
python
node
pkgs.nodePackages.yarn
];
shellHook = ''
export RUBY_VERSION="$(ruby -e 'puts RUBY_VERSION.gsub(/\d+$/, "0")')"
export GEM_HOME="$(pwd)/vendor/bundle/ruby/$RUBY_VERSION"
export BUNDLE_PATH="$(pwd)/vendor/bundle"
export PATH="$GEM_HOME/bin:$PATH"
'';
}
python/pyenv/venv: {
pkgs ? import <nixpkgs> {},
}:
let
python_packages = python-packages: [
python-packages.pip
];
python = pkgs.python38.withPackages python_packages;
in pkgs.mkShell {
buildInputs = [
python
];
shellHook = ''
export PYTHON_VERSION="$(python -c 'import platform; import re; print(re.sub(r"\.\d+$", "", platform.python_version()))')"
export PIP_PREFIX="$PWD/vendor/python/$PYTHON_VERSION/packages"
export PYTHONPATH="$PIP_PREFIX/lib/python$PYTHON_VERSION/site-packages:$PYTHONPATH"
unset SOURCE_DATE_EPOCH
export PATH="$PIP_PREFIX/bin:$PATH"
'';
}
mixing arm64 and x86_64 on an Apple Silicon machine: {
x86_64 ? import <nixpkgs> { localSystem = "aarch64-darwin"; },
aarch64 ? import <nixpkgs> { localSystem = "x86_64-darwin"; }
}:
let
foo = aarch64.foo;
in aarch64.mkShell { # this makes nix-shell drop to an arm64 shell, change it to x86_64 to be intel/Rosetta2
buildInputs = [
foo
x86_64.bar
aarch64.baz
];
}
using an unstable/pinned/git package: {
stable ? import <nixpkgs> {},
unstable ? import (fetchTarball http://nixos.org/channels/nixos-unstable/nixexprs.tar.xz) {},
pinned ? import (fetchTarball https://github.com/nixos/nixpkgs/archive/ca2ba44cab47767c8127d1c8633e2b581644eb8f.tar.gz) {},
git ? import (fetchGit { url = "https://github.com/nixos/nixpkgs/"; ref = "refs/heads/nixos-unstable"; rev = "ca2ba44cab47767c8127d1c8633e2b581644eb8f"; }) {},
}:
let
foo = stable.foo;
in stable.mkShell {
buildInputs = [
foo
unstable.bar
pinned.baz
git.qux
];
}
selecting a particular C/C++ compiler&stdlibc++ version: {
pkgs ? import <nixpkgs> {},
}:
let
clang = pkgs.clang_12
in pkgs.llvmPackages_12.stdenv.mkDerivation {
buildInputs = [
clang
];
}
The trap is: You could handle all of that by using or writing nix features. You could even use NixOS instead of whatever distro you're used to. But then by going cold-turkey you have to learn whatever nix feature on top of all the basic (as in fundamental) things nix has to offer. Purists would say "no no no this is not the sanctioned way", which is kind of true but also setting yourself up for failure; it'd be like looking at a mountain and trying to jump right to the top, which of course you will fail to, when you could just be climbing it and be successful. Whatever practical gets you on board is fine. You can get to the "pure nix" stuff later, if you ever need to.Thanks to a chatty Romanian taxi driver I had earlier today, I understand this.
I recently gave nix a try because of a hacker news comment and agree with most of the points in this comment. The fact that new users are encouraged to read not 1, but 3 separate manuals to grok the tool is extremely discouraging for new users. Also the fact that nix is currently transitioning to a new feature called flakes and deprecating channels signals to me that the ecosystem is not stable enough for me to recommend that my team at work use it right now.
That being said, I intend to read at least 2 of the manuals and use nix for dependency management in my side projects and to manage my system dependencies. I think nix takes all the great things that asdf can do, to a whole new level. The dream of having a nix section in my projects 'Getting Started' that only has 2 steps (1. Install nix 2. Run `nix-shell`) is something I desperately want.
I think nix is much like erlang in that it's incredibly powerful and solves a lot of common problems, but it's used and maintained by an old guard that went through the trouble of learning how to use it and maintains the "I did it, why can't you" mindset. The erlang ecosystem has been given new life and been made easily available to new users thanks to the awesome work done by Jose Valim and the rest of the elixir team and community. However, I really hope that it doesn't take 30 years and a new language built on top of nix for it to become accessible to everyone.
For those working on nix and the new flake functionality please look to things like elixir and asdf for inspiration on documentation, ease of use and pragmatism. Help make the tool you love ubiquitous. It will ultimately make your life easier when working with new developers because they will already have nix installed.
I intended to do everything I can to help push nix in this direction (once I understand how the hell it works) if anyone reading this, has some time on their hands and wants to improve the state of software development for everyone please go down the nix rabbit hole and try to find ways to contribute to making it more accessible.
I've found it to be quite flexible though. For example, here's a commit in which I apply a patch to a tool to solve a problem that the derivation hadn't taken into account (and absent a home-manager solution): https://github.com/samhh/dotfiles/commit/867dd3b4d4b3942a0aa...
I understand everything in that sentence except the word "so".
I really like Nix but in this particular way it feels like taking a big step backwards from the other languages I frequently use, particularly for a language in which you'll necessarily be constantly interacting with bespoke interfaces.
I guess knowing allowed types for each parameter is informative (more if very specific types are used rather than string or integer), but documentation usually specifies that anyway. And even knowing all the types isn't usually enough without documentation.
My point is just that I've used plenty of libraries in dynamically typed languages without needing to read the source. And conversely, I've occasionally needed to look at the source of a function in a staticly typed language, to answer some question not answered by the types and documentation.
Haskell is the ideal here, since it separates the type signature into its own separate line at the top of the function declaration. That confers readability and reduces cognitive overhead more than any other language I've used.
It also changes the programming thought process. You can pseudo-code an entire program just with function type definitions. Then test that the type defs compile without error, and go back and implement the function definitions. As long as the function definitions adhere to their type signature, the program almost always works (barring some I/O errors).
Not all statically typed languages are created equal. The ones fundamentally oriented around function type signatures, rather than variable types, are the ones I think parent had in mind in his comment.
Really, I like the idea and I actually like the fact that they use Guile (scheme) instead of a home-built DSL, but guix is a horrible, poorly documented, doesn't work even on the happy path mess.
But maybe I'm just a "GNU zealot".
I didn't mean to imply everyone who uses it is a GNU zealot, but there's definitely a subset who are.
Look at Python with Nix: https://nixos.wiki/wiki/Python And then consider the simplicity of `pyenv install x.x.x` and a requirements.txt.
"possible future efficiency gains".
With this in mind, the question shifts from rather vague talk about possible futures to various kinds of scenarios.
What happens when software developers have a deterministic build system? On the whole, the benefits are significant. Sure, there are costs to get there. In my view, the benefits often outweigh the costs.
I disagree with the whole "highly likely" bit. Just because something new has a large positive value, that does not mean that a large time-investment or effort-investment in it is likely to pay off.
After all, it may never get traction to make the cost of learning it worthwhile - some other, newer, better, easier system could come along and be adopted instead.
We've seen this play out again and again in the technology world[1]. In 2000, a large time and effort investment into UTF-16 would be wasted as the world adopted UTF-8 instead. A large time and effort invested into new languages with powerful features (Haskell, circa 2005) would be wasted as the industry largely ignored it in favour of (presumably) inferior alternatives.
Is nix highly likely to provide solutions to a whole class of problems?
Sure.
Is it highly likely to be the dominant, or even a common tool, in the future that solves those problems?
Doubtful; "easier-to-use" has always won against "can better solve problems". I see no evidence that this will change anytime soon.
[1] I've found that by procrastinating about learning a new technology, I can sometimes manage to avoid having to learn it altogether, because the tech dies off before my procrastination does. By putting off learning VRML in 2001, it went away before I ever got to it.
Likewise, I'll put off learning new things until I actually have a need for the solution they offer. At least then, even if they die out in a few years, I'd have at least solved my immediate problems.
Your choice of language: saying 'need' suggests a black and white decision. See what I mean?
Costs and benefits often cannot be estimated accurately 'in one go'. It is common to spend some time and money to increase understanding and thus reduce uncertainty.
What deterministic build systems have you used? If you have not, how confident should you be that you can accurately assess their costs and benefits?
PS. Your comments suggest to me that, like most people with experience, you have become overconfident. To disabuse yourself of this, I suggest you find a prediction market and make some bets. You'll find you have to make your claims testable. Make a specific prediction/bet on Nix and let's see what happens. Let us know how it goes.
The way you've framed the question is notable. The argument structure ignores a common pattern in software: tools steal ideas from each other often.
For example, MySQL is very different than its 2002 version. There are more database back ends with various guarantees and performance tradeoffs.
You are disagreeing with something I didn't write.
I keep seeing nix mentioned alongside complaints about it's maturity but that seems par for the course for something new, doesn't it? Until a tool reaches critical mass sharp edges are to be expected.
I'll look into nix again when I have an appropriate use case.
More broadly, even if it isn't as popular, Nix is a viable tool and also encourages us (developers) to seek out repeatability in our dependencies.
Be prepared to sink dozens and dozens of hours learning the language, it’s features, and switching everything over. Be prepared to get stuck when you are required to use an old version of a language and things don’t install correctly — and sometimes blocked by nix itself from installing it. Be prepared to get stuck if you are not in the happy path. Be prepared to get lost in the sometimes scant documentation.
I love the idea of nix, but it did not work for me except for the simplest cases.