News for hackers. Stuff that matters.
Re:Nix is the ultimate DevOps toolkit by Shoue on Friday April 09, 2021 @08:58AM (#26749967)
I wish Nix would just get rid of nix-env. It's not the way it's meant to be used, you should be using "nix run" or nix-shell for temporary usage, and home-manager for dotfiles and user dependencies. Using Nix like an imperative package manager is not really an improvement over existing ones, the declarative bit is where it truly shines.
Re:Nix is the ultimate DevOps toolkit by andrewla on Friday April 09, 2021 @09:32AM (#26750327)
Hear hear! The desire to make the experience "familiar" for users of apt causes way more confusion once you start actively using it.
Re:Nix is the ultimate DevOps toolkit by gmfawcett on Friday April 09, 2021 @12:53PM (#26752907)
Eh, I don't think nix-env is all that bad. Put it this way:

nix-shell lets you dip your toes in the water, without learning the Nix language. Amazing value added... but sometimes it's annoying that it forgets that package you used yesterday.

nix-env lets you "install" software, without learning the Nix language. Great, now I can use that Nix package repeatedly without having to nix-shell it. We're still better off than "apt install" at this point, and it's no harder than that.

Everything else -- nixos config, home-manager, flakes, overlays, etc. -- requires Nix language knowledge, and non-superficial comfort level with the Nix data model (nixpkgs, attribute sets, and a very long tail of other things).

Nix-env is a beneficial stepping stone for converting dabblers into diehards, and is one of the Nash equilibria for some Nix users. (i.e., not everyone wants to learn the language and drink the koolaid, they just want to enjoy some additional benefits beyond nix-shell.)

Re:Nix is the ultimate DevOps toolkit by jonringer117 on Friday April 09, 2021 @02:41PM (#26754348)
The issue with `nix-env` is how it tries to decide a later version. Python also has this issue, because it will see that python3.7 has a newer version called `python3.10a3`, and installs the alpha/dev version, instead of the expected python3.7.x

Honestly, if people just used nix for `shell.nix` in projects, it's still major value added

Re:Nix is the ultimate DevOps toolkit by gmfawcett on Friday April 09, 2021 @03:16PM (#26754838)
Is there a tool that works something like this:

- mkdir myproject ; cd myproject

- nix-shell-env init # creates an "empty-ish" shell.nix

- nix-shell-env install nixpkgs.whatever # adds dep. to shell.nix

- nix-shell # activate it

So you don't have to learn any Nix-language / nixpkgs ropes, but can still have a viable shell.nix built for your project. I could see that being a nice stepping stone between "nix-shell -p" and "learning the language."

Re:Nix is the ultimate DevOps toolkit by chaorace on Friday April 09, 2021 @09:13AM (#26750105)
Agreed, but perhaps they should officially adopt home-manager then? It felt strange to me that the best way to be using nix was a separate community project.
Re:Nix is the ultimate DevOps toolkit by tnzm on Friday April 09, 2021 @09:16AM (#26750140)
I don't use nix-env or home-manager. On my NixOS workstation everything (outside of nix-shells) is installed via environment.systemPackages. And I do wish there was an easy way to add stuff to install packages without rebuilding the whole system - I guess that's what nix-env is for but if I went with that I would the "single editable config file that you can put in VCS" aspect. I think nix-env uses profiles, which are implemented collection of symlinks to the Nix store - guess I could still version those...?
Re:Nix is the ultimate DevOps toolkit by danieldk on Friday April 09, 2021 @09:38AM (#26750402)
Re:Nix is the ultimate DevOps toolkit by chriswarbo on Friday April 09, 2021 @09:42AM (#26750446)
You can define a package that contains a bunch of other things, e.g. I use packages like 'devCli', 'netGui', etc. (I actually put these in my systemPackages on NixOS and nix-darwin, but they would also work with nix-env)

http://chriswarbo.net/git/nix-config/git/branches/master/ove...

Re:Nix is the ultimate DevOps toolkit by otabdeveloper4 on Friday April 09, 2021 @10:44AM (#26751181)
At the moment the happy path for Nix is using it in the same space that Docker is used now - for providing reproducible dev/prod environments for server backends. (And in fact Docker and Nix complement and play nice with each other.)

You could theoretically use Nix for managing user apps too, but that's a niche use case and (as you found out) not in a working state.

Re:Nix is the ultimate DevOps toolkit by jpe90 on Friday April 09, 2021 @11:14AM (#26751547)
So for the happy path, are you declaring the entire development toolchain on a project by project basis? Any chance you have some recommended resources that show the way you're using it? I was hoping to use it for Haskell but I'd love to see a guide targeted to new users that shows what you're describing.
Re:Nix is the ultimate DevOps toolkit by otabdeveloper4 on Friday April 09, 2021 @03:07PM (#26754710)
Yeah, each project has its own default.nix that might look something like this:

    {
      pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/cd63096.tar.gz") { }
    } :
    with pkgs;
    let
      mypackage = callPackage ./mypackage.nix { stdenv = stdenv; python = python3; };
    in
    dockerTools.buildImage {
      contents = [ mypackage ];
    }
You then write a package definition in mypackage.nix, following the template of packaging things in nixpkgs.

(Dev environments work in a similar fashion except with mkShell instead of dockerTools.buildImage.)

I haven't used Nix flakes but I think it develops this simple idea further.

Re:Nix is the ultimate DevOps toolkit by epistasis on Friday April 09, 2021 @11:40AM (#26751961)
As somebody who wants to try out nix, but has almost no knowledge of it, I would like to find resources that show the "right" way, or perhaps the "best" way to use nix. Do you know of existing documentation that explains that, according to your opinion?
Re:Nix is the ultimate DevOps toolkit by 2mol on Friday April 09, 2021 @11:46AM (#26752055)
I think https://nix.dev is what you are looking for. It's a really great resource (and no need to go all in and do _everything_ in nix)
Re:Nix is the ultimate DevOps toolkit by jpe90 on Friday April 09, 2021 @12:03PM (#26752275)
This is orders of magnitude more useful than the official docs.
Re:Nix is the ultimate DevOps toolkit by nextos on Friday April 09, 2021 @02:14PM (#26753957)
The unofficial wiki is also very helpful: https://nixos.wiki/
Re:Nix is the ultimate DevOps toolkit by danmur on Friday April 09, 2021 @10:04PM (#26758324)
Ahh that's great, thank you. I've been running nixos on my laptop for a year but have never come across that somehow.
Re:Nix is the ultimate DevOps toolkit by jonringer117 on Friday April 09, 2021 @02:43PM (#26754368)
I made a few videos, https://www.youtube.com/channel/UC-cY3DcYladGdFQWIKL90SQ . However, I need to produce more and of higher quality.
Re:Nix is the ultimate DevOps toolkit by domenkozar on Friday April 09, 2021 @09:39AM (#26750414)
That's the long term plan, once the new command line stabilizes.
Re:Nix is the ultimate DevOps toolkit by throwaway894345 on Friday April 09, 2021 @10:25AM (#26750970)
I agree, but I think nix-env is only the tip of the iceberg with respect to bad upfront experiences. If you try to make a Go package, for example, you end up having to provide a hash. To generate the hash `vendorSha256`, you have to run `nix-prefetch`, but that program is broken on MacOS. I mentioned elsewhere that I spent a weekend working with the Nix discord to try to install VS Code with some extensions for Rust development, and we collectively couldn't figure it out (I'm not using NixOS, but just Nix the package manager). In the past I've tried to package a simple Python app thinking it would be a simple reference to a pypi package, and I ended up having to package a never-ending matrix of the most obscure C dependencies and ultimately gave up. Maybe if I were doing Haskell development I might find the Nix happy path, but for now I just have a nix shell that installs a few basic CLI utilities and that's the only way I've managed to use Nix successfully. Further, any time I have to do anything in nixpkgs, I find that everything is a slog--since there are so few imports (and because the Nix people don't believe in documentation or types or even sane file/directory organization), in order to find the "shape" of argument `foo` to the Nix package function `Bar`, I need to grep the whole nixpkgs repo for something that uses `Bar` and then figure out what it's passing in for the `foo` parameter. Often `Bar` is passed to several other functions (defined in several other files) before it's called and the thing that gets passed in for the `foo` parameter might be the return value of a function which itself is returning the result of some other function (each of which is defined in other files), etc. This results in a combinatorial explosion of inefficiently grepping around the codebase just to get the type of a symbol (something that would come for free in a statically typed language).

I'm a big believer in the Nix approach to builds, and that this approach holds a lot of potential for DevOps (especially CI/CD); however, there are so many practical problems with Nix and the community doesn't seem especially interested in fixing them. Of course, what volunteers spend their time on is there prerogative, I could volunteer my time rather than criticize, shame on me, etc but it's a loss for the broader world. Personally I would really like to see a community with the ethos of the Rust community pick up the torch and try to build something inspired by Nix, but perhaps with more pragmatic choices (e.g., instead of inventing an obscure scripting language, they might use something more familiar and thus easier on new users; rather than the chaos of nixpkgs, they might give users a type system and/or good editor tooling; rather than the difficult-to-Google "Nix" label, they might pick something a bit less overloaded; etc).

Re:Nix is the ultimate DevOps toolkit by rkrzr on Friday April 09, 2021 @10:46AM (#26751202)
(author here) I agree that Nix not being statically typed is one of the biggest drawbacks of the language. Having it statically typed with a strong type system like Haskell's would help solve some of the readability issues that you mention. Types of course also serve as documentation, which is another often-mentioned weak point of the language. Finally, with static types we could also have much better editor support with nice things like auto-completion.

Unfortunately, I'm not aware of any efforts to add (optional) typing to the Nix language.

Re:Nix is the ultimate DevOps toolkit by throwaway894345 on Friday April 09, 2021 @11:03AM (#26751414)
I think [Nickel][1] aims to add static types to Nix, but I don't know of any efforts to integrate Nickel into nixpkgs.

[1]: https://github.com/tweag/nickel

Re:Nix is the ultimate DevOps toolkit by jonringer117 on Friday April 09, 2021 @02:44PM (#26754396)
It's still in development. But swapping a language is possible, as it's only needed for evaluation. Building the package (realization) just needs the derivation.
Re:Nix is the ultimate DevOps toolkit by camehere3saydis on Friday April 09, 2021 @07:28PM (#26757429)
So does that mean I can just hack together my own derivations in, say, Python, and bypass the whole nix-shell vs. nix-env, overlays vs overrides, home-manager, flakes, etc. menagerie? And use only Nix only for its central artifact store and dependency resolution?

It kind of looks like they're trying out different interfaces to the same basic core functionality and seeing what catches.

My personal gripe is that wrappers for languages that have their own package managers are abysmal. But griping about Nix on HN is just extra publicity. Now we're just waiting for someone to explain it in a sufficiently succinct way for the crowd.

Come to think of it, that's kind of like Monads in Haskell a few years back. Anyone remember the series of watered down explanations that failed to address the actual problem people were having, which was that that people were confused by the notation, if I recall correctly.

Nix's roots are close to Haskell turf, so it comes as no surprise they're struggling with explaining their abstractions, resulting in the series of mutually overlapping "official" ways to do things (none of which, incidentally, is the "obvious" way if there is one). After all, Nix is a build tool and not a whole language, so they get to try different CLIs and config APIs within with less overhead than experimenting with a language's syntax.

Re:Nix is the ultimate DevOps toolkit by danieldk on Friday April 09, 2021 @11:12AM (#26751524)
If you try to make a Go package, for example, you end up having to provide a hash. To generate the hash `vendorSha256`, you have to run `nix-prefetch`, but that program is broken on MacOS.

You can also stub the hash (e.g. with lib.fakeSha256). nix-build will fail with a hash mismatch and tell you the correct hash.

I mentioned elsewhere that I spent a weekend working with the Nix discord to try to install VS Code with some extensions for Rust development,

That's tricky with e.g. IntelliJ as well. For development, I now just use rustup and use nix-shell/nix devshell to define native dependencies. Outside development I do full Nix builds with crate2nix or Naersk.

In the past I've tried to package a simple Python app thinking it would be a simple reference to a pypi package, and I ended up having to package a never-ending matrix of the most obscure C dependencies and ultimately gave up.

Python in Nix is painful for various reasons. One is that since the Python interpreter cannot handle multiple versions of a package, nixpkgs can only have one version of each package. However, since many Python packages do not follow semver, it often requires a lot of patching. And then there are issues like Python packages writing directly to their package directory (which does not work with a read-only store). I think packaging Python packages with Nix is worthwhile, but for Python development I think it is easier to just build an FHS environment and use regular Python packaging tools (pip, Poetry, etc.).

This is an issue with some other language ecosystems as well, their packaging models are incompatible with Nix.

This results in a combinatorial explosion of inefficiently grepping around the codebase just to get the type of a symbol*

nix repl* has helped me in many cases. But I agree that Nix would be so much nicer with static typing.

however, there are so many practical problems with Nix and the community doesn't seem especially interested in fixing them

I think most of us are interested in fixing problems. It's just that nixpkgs is large and people have deployed various Nix versions. So, it is quite hard to make fundamental changes to Nix (e.g. typing) and roll them out. Things are happening, e.g. flakes provide a standard interface to package sets, there is work on supporting content-adressed outputs (outside fixed-output derivations), but with the size of the Nix ecosystem, these things take a long time to crystallize and roll out.

Re:Nix is the ultimate DevOps toolkit by throwaway894345 on Friday April 09, 2021 @01:37PM (#26753518)
Thanks for taking the time to respond. It's good to know that there are reasons/workarounds for the various problems, but I hope you understood my main thrust, which is not that this is an exhaustive list of issues with Nix, but rather these are the kinds of issues one can expect to encounter (or perhaps I'm just unusually unlucky) when delving into Nix.

I get the perception that people think Nix is a panacea for common CI/CD problems (and to be clear, I think it could be such a panacea one day) but rather it solves some problems but introduces many more, harder problems--which no doubt individually have workarounds and so on, but finding them probably requires intimate knowledge of Nix and the ecosystem that you're trying to package. Unfortunately, I can live with suboptimal builds and a little nondeterminism/impurity, but I can't be dead-in-the-water because I can't figure out how to package a dependency that my app depends on.

So I'm rooting hard for Nix, but I want to share my experiences so potential users know what they might run into and so any Nix contributors know a bit more what users are running into (if it's not already known).

> I think most of us are interested in fixing problems.

I didn't mean to imply you weren't interested in fixing any problems, but rather the problems that I care about, which have persisted since I tried Nix for the first time circa 2014.

> So, it is quite hard to make fundamental changes to Nix (e.g. typing) and roll them out.

Surely gradual typing would be relatively easy to roll out, as the name implies? Similarly, documentation improvements could be rolled out pretty easily (e.g., put beginners on the happy path, don't recommend nix-env, tell people about homemanager if that's the thing they should be using, etc). I mean these things in the spirit of constructive feedback--if you want to court new users, these seem like low hanging fruit.

> Things are happening, e.g. flakes provide a standard interface to package sets, there is work on supporting content-adressed outputs (outside fixed-output derivations), but with the size of the Nix ecosystem, these things take a long time to crystallize and roll out.

I'm glad to hear things are moving--I'll need to read up on content-addressed output and flakes!

Re:Nix is the ultimate DevOps toolkit by mdoms on Friday April 09, 2021 @12:47PM (#26752818)
> You can also stub the hash (e.g. with lib.fakeSha256). nix-build will fail with a hash mismatch and tell you the correct hash.

Are you kidding me?

Re:Nix is the ultimate DevOps toolkit by soraminazuki on Friday April 09, 2021 @10:31PM (#26758438)
What do you mean? This is exactly how hash verification is supposed to work. You either supply the correct hash or the verification fails, telling you the actual and expected hashes. Works the same way on any package manager that verifies its sources.
Re:Nix is the ultimate DevOps toolkit by jamie_ca on Friday April 09, 2021 @01:43PM (#26753587)
Oh, shades of my favorite bug ever, doing commercial product QA two decades ago.

The product had an external license file you'd need to drop alongside, and on boot if your licence key didn't match for your name + connection settings, the retail build would spit out both the key in the file, and the expected correct key for your settings.

It was not a priority fix for them, either. At least this one is merely informational, and not a potential gate on revenue.

Re:Nix is the ultimate DevOps toolkit by takeda on Friday April 09, 2021 @03:42PM (#26755139)
In this case it is intentional and basically became a feature to the point that probably majority uses.

The hash requirement is kind of a chicken & egg problem, because to fetch a file you need the hash, and to get the hash you need to fetch the file. So this gives a way to escape it.

There's also the "proper way" to use the prefetch command (I personally like the nix-universal-prefetch) which basically fetches file to the cache and outputs the hash, but in that case it is one extra command which you need to figure arguments for, so the former method is just faster.

Re:Nix is the ultimate DevOps toolkit by jonringer117 on Friday April 09, 2021 @02:47PM (#26754442)
It's content addressable. If you use the old hash, it will just use the "old" contents happily (unless they don't exist). So you have to alter it slightly to get a "cache miss" to see what should have been produced.

This can be address by adding the version to the dependencies store path. This has been done for rust builds, not sure about go.

Re:Nix is the ultimate DevOps toolkit by Ericson2314 on Friday April 09, 2021 @11:49AM (#26752095)
At the very least, get rid of `-u`. Some parts of nix-env are used underneath the hood for NixOS and home-manager I beleive, but not that.
Re:Nix is the ultimate DevOps toolkit by jonringer117 on Friday April 09, 2021 @02:39PM (#26754313)
at least now, people get recommend to use `nix-shell -p <pkg>` when something is missing in NixOS