News for hackers. Stuff that matters.
Re:Nix is the ultimate DevOps toolkit by throwaway894345 on Friday April 09, 2021 @10:25AM (#26750970)
I agree, but I think nix-env is only the tip of the iceberg with respect to bad upfront experiences. If you try to make a Go package, for example, you end up having to provide a hash. To generate the hash `vendorSha256`, you have to run `nix-prefetch`, but that program is broken on MacOS. I mentioned elsewhere that I spent a weekend working with the Nix discord to try to install VS Code with some extensions for Rust development, and we collectively couldn't figure it out (I'm not using NixOS, but just Nix the package manager). In the past I've tried to package a simple Python app thinking it would be a simple reference to a pypi package, and I ended up having to package a never-ending matrix of the most obscure C dependencies and ultimately gave up. Maybe if I were doing Haskell development I might find the Nix happy path, but for now I just have a nix shell that installs a few basic CLI utilities and that's the only way I've managed to use Nix successfully. Further, any time I have to do anything in nixpkgs, I find that everything is a slog--since there are so few imports (and because the Nix people don't believe in documentation or types or even sane file/directory organization), in order to find the "shape" of argument `foo` to the Nix package function `Bar`, I need to grep the whole nixpkgs repo for something that uses `Bar` and then figure out what it's passing in for the `foo` parameter. Often `Bar` is passed to several other functions (defined in several other files) before it's called and the thing that gets passed in for the `foo` parameter might be the return value of a function which itself is returning the result of some other function (each of which is defined in other files), etc. This results in a combinatorial explosion of inefficiently grepping around the codebase just to get the type of a symbol (something that would come for free in a statically typed language).

I'm a big believer in the Nix approach to builds, and that this approach holds a lot of potential for DevOps (especially CI/CD); however, there are so many practical problems with Nix and the community doesn't seem especially interested in fixing them. Of course, what volunteers spend their time on is there prerogative, I could volunteer my time rather than criticize, shame on me, etc but it's a loss for the broader world. Personally I would really like to see a community with the ethos of the Rust community pick up the torch and try to build something inspired by Nix, but perhaps with more pragmatic choices (e.g., instead of inventing an obscure scripting language, they might use something more familiar and thus easier on new users; rather than the chaos of nixpkgs, they might give users a type system and/or good editor tooling; rather than the difficult-to-Google "Nix" label, they might pick something a bit less overloaded; etc).

Re:Nix is the ultimate DevOps toolkit by rkrzr on Friday April 09, 2021 @10:46AM (#26751202)
(author here) I agree that Nix not being statically typed is one of the biggest drawbacks of the language. Having it statically typed with a strong type system like Haskell's would help solve some of the readability issues that you mention. Types of course also serve as documentation, which is another often-mentioned weak point of the language. Finally, with static types we could also have much better editor support with nice things like auto-completion.

Unfortunately, I'm not aware of any efforts to add (optional) typing to the Nix language.

Re:Nix is the ultimate DevOps toolkit by throwaway894345 on Friday April 09, 2021 @11:03AM (#26751414)
I think [Nickel][1] aims to add static types to Nix, but I don't know of any efforts to integrate Nickel into nixpkgs.

[1]: https://github.com/tweag/nickel

Re:Nix is the ultimate DevOps toolkit by jonringer117 on Friday April 09, 2021 @02:44PM (#26754396)
It's still in development. But swapping a language is possible, as it's only needed for evaluation. Building the package (realization) just needs the derivation.
Re:Nix is the ultimate DevOps toolkit by camehere3saydis on Friday April 09, 2021 @07:28PM (#26757429)
So does that mean I can just hack together my own derivations in, say, Python, and bypass the whole nix-shell vs. nix-env, overlays vs overrides, home-manager, flakes, etc. menagerie? And use only Nix only for its central artifact store and dependency resolution?

It kind of looks like they're trying out different interfaces to the same basic core functionality and seeing what catches.

My personal gripe is that wrappers for languages that have their own package managers are abysmal. But griping about Nix on HN is just extra publicity. Now we're just waiting for someone to explain it in a sufficiently succinct way for the crowd.

Come to think of it, that's kind of like Monads in Haskell a few years back. Anyone remember the series of watered down explanations that failed to address the actual problem people were having, which was that that people were confused by the notation, if I recall correctly.

Nix's roots are close to Haskell turf, so it comes as no surprise they're struggling with explaining their abstractions, resulting in the series of mutually overlapping "official" ways to do things (none of which, incidentally, is the "obvious" way if there is one). After all, Nix is a build tool and not a whole language, so they get to try different CLIs and config APIs within with less overhead than experimenting with a language's syntax.

Re:Nix is the ultimate DevOps toolkit by danieldk on Friday April 09, 2021 @11:12AM (#26751524)
If you try to make a Go package, for example, you end up having to provide a hash. To generate the hash `vendorSha256`, you have to run `nix-prefetch`, but that program is broken on MacOS.

You can also stub the hash (e.g. with lib.fakeSha256). nix-build will fail with a hash mismatch and tell you the correct hash.

I mentioned elsewhere that I spent a weekend working with the Nix discord to try to install VS Code with some extensions for Rust development,

That's tricky with e.g. IntelliJ as well. For development, I now just use rustup and use nix-shell/nix devshell to define native dependencies. Outside development I do full Nix builds with crate2nix or Naersk.

In the past I've tried to package a simple Python app thinking it would be a simple reference to a pypi package, and I ended up having to package a never-ending matrix of the most obscure C dependencies and ultimately gave up.

Python in Nix is painful for various reasons. One is that since the Python interpreter cannot handle multiple versions of a package, nixpkgs can only have one version of each package. However, since many Python packages do not follow semver, it often requires a lot of patching. And then there are issues like Python packages writing directly to their package directory (which does not work with a read-only store). I think packaging Python packages with Nix is worthwhile, but for Python development I think it is easier to just build an FHS environment and use regular Python packaging tools (pip, Poetry, etc.).

This is an issue with some other language ecosystems as well, their packaging models are incompatible with Nix.

This results in a combinatorial explosion of inefficiently grepping around the codebase just to get the type of a symbol*

nix repl* has helped me in many cases. But I agree that Nix would be so much nicer with static typing.

however, there are so many practical problems with Nix and the community doesn't seem especially interested in fixing them

I think most of us are interested in fixing problems. It's just that nixpkgs is large and people have deployed various Nix versions. So, it is quite hard to make fundamental changes to Nix (e.g. typing) and roll them out. Things are happening, e.g. flakes provide a standard interface to package sets, there is work on supporting content-adressed outputs (outside fixed-output derivations), but with the size of the Nix ecosystem, these things take a long time to crystallize and roll out.

Re:Nix is the ultimate DevOps toolkit by throwaway894345 on Friday April 09, 2021 @01:37PM (#26753518)
Thanks for taking the time to respond. It's good to know that there are reasons/workarounds for the various problems, but I hope you understood my main thrust, which is not that this is an exhaustive list of issues with Nix, but rather these are the kinds of issues one can expect to encounter (or perhaps I'm just unusually unlucky) when delving into Nix.

I get the perception that people think Nix is a panacea for common CI/CD problems (and to be clear, I think it could be such a panacea one day) but rather it solves some problems but introduces many more, harder problems--which no doubt individually have workarounds and so on, but finding them probably requires intimate knowledge of Nix and the ecosystem that you're trying to package. Unfortunately, I can live with suboptimal builds and a little nondeterminism/impurity, but I can't be dead-in-the-water because I can't figure out how to package a dependency that my app depends on.

So I'm rooting hard for Nix, but I want to share my experiences so potential users know what they might run into and so any Nix contributors know a bit more what users are running into (if it's not already known).

> I think most of us are interested in fixing problems.

I didn't mean to imply you weren't interested in fixing any problems, but rather the problems that I care about, which have persisted since I tried Nix for the first time circa 2014.

> So, it is quite hard to make fundamental changes to Nix (e.g. typing) and roll them out.

Surely gradual typing would be relatively easy to roll out, as the name implies? Similarly, documentation improvements could be rolled out pretty easily (e.g., put beginners on the happy path, don't recommend nix-env, tell people about homemanager if that's the thing they should be using, etc). I mean these things in the spirit of constructive feedback--if you want to court new users, these seem like low hanging fruit.

> Things are happening, e.g. flakes provide a standard interface to package sets, there is work on supporting content-adressed outputs (outside fixed-output derivations), but with the size of the Nix ecosystem, these things take a long time to crystallize and roll out.

I'm glad to hear things are moving--I'll need to read up on content-addressed output and flakes!

Re:Nix is the ultimate DevOps toolkit by mdoms on Friday April 09, 2021 @12:47PM (#26752818)
> You can also stub the hash (e.g. with lib.fakeSha256). nix-build will fail with a hash mismatch and tell you the correct hash.

Are you kidding me?

Re:Nix is the ultimate DevOps toolkit by soraminazuki on Friday April 09, 2021 @10:31PM (#26758438)
What do you mean? This is exactly how hash verification is supposed to work. You either supply the correct hash or the verification fails, telling you the actual and expected hashes. Works the same way on any package manager that verifies its sources.
Re:Nix is the ultimate DevOps toolkit by jamie_ca on Friday April 09, 2021 @01:43PM (#26753587)
Oh, shades of my favorite bug ever, doing commercial product QA two decades ago.

The product had an external license file you'd need to drop alongside, and on boot if your licence key didn't match for your name + connection settings, the retail build would spit out both the key in the file, and the expected correct key for your settings.

It was not a priority fix for them, either. At least this one is merely informational, and not a potential gate on revenue.

Re:Nix is the ultimate DevOps toolkit by takeda on Friday April 09, 2021 @03:42PM (#26755139)
In this case it is intentional and basically became a feature to the point that probably majority uses.

The hash requirement is kind of a chicken & egg problem, because to fetch a file you need the hash, and to get the hash you need to fetch the file. So this gives a way to escape it.

There's also the "proper way" to use the prefetch command (I personally like the nix-universal-prefetch) which basically fetches file to the cache and outputs the hash, but in that case it is one extra command which you need to figure arguments for, so the former method is just faster.

Re:Nix is the ultimate DevOps toolkit by jonringer117 on Friday April 09, 2021 @02:47PM (#26754442)
It's content addressable. If you use the old hash, it will just use the "old" contents happily (unless they don't exist). So you have to alter it slightly to get a "cache miss" to see what should have been produced.

This can be address by adding the version to the dependencies store path. This has been done for rust builds, not sure about go.