News for hackers. Stuff that matters.
Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by jljljl on Thursday May 25, 2023 @04:29PM (#36075766)
Finding a specific version of a package is also really difficult. It requires you to find the commit hash in the Nixpkgs repo that last contained it, and there isn't a central search engine that lets you map version to commit.

We ended up building package version search into the Devbox CLI for this reason (https://www.jetpack.io/devbox/docs/guides/pinning_packages/#...). There's also a 3rd party site that lets you search for Nix package versions (https://lazamar.co.uk/nix-versions/)

Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by peter_l_downs on Thursday May 25, 2023 @05:16PM (#36076334)
I'll have to check out devbox, thanks for the link. I don't understand why Nix doesn't have some way for derivations to claim that they install certain artifacts, which would make searching so much easier.

The version thing is also really annoying, thanks for the helpful link, I hadn't seen that. Not sure how this will work in a Flakes world. Seems weird to me that sometimes packages are important enough to have multiple derivations each installing a different version (like the pythonFull packages) but there is no standard/good/default solution for it.

Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by jljljl on Thursday May 25, 2023 @05:42PM (#36076628)
Versioning is a bit easier with Flakes because they're generally published in their own Git repository, meaning you can use tags or branch refs to publish/install specific versions of a package. You can usually do something like `github:org/repo/ref#package` to get what you need.
Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by peter_l_downs on Thursday May 25, 2023 @05:51PM (#36076729)
Yes, but how will this work for the `nixpkgs` repository? Still very unclear!

I'm 100% onboard with flakes in general, and use flakes to describe devshells + publish binaries when applicable -- like with the tool I linked in my earlier post.

Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by NathanOsullivan on Thursday May 25, 2023 @05:24PM (#36076431)
Normally I would eye-roll at these kind of self promotions but it looks like you are trying to help with a real pain point of Nix.

I put a good amount of time getting to grips with "raw" nix with the I imagine common yo-yo-ing between "I don't get it" and "oh I see, this is great" but when I realised how the intersection of nixpkgs and package versioning actually worked.. I was done.

For a tool that from the outside seems is so heavily focused on immutability to just continually throw away old versions in nixpkgs head is a head scratcher, and as a monorepo isn't a great fit for utilising different revisions for different packages either.

I can only guess that due to single repo containing every package it wasn't seen as practical to just continually append versions to, but when the diff log is just full of 'delete version X URL and it's hash, add X+1 and new hash' from the outside at least, it felt like a real missed opportunity.

Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by viraptor on Thursday May 25, 2023 @05:40PM (#36076614)
I would suggest one more step of "oh, this is great": I don't think people need to care as much about finding the right version in the repo as they do. (Unless we're talking about something really significant like finding old MySQL 5.6) For smaller apps you can either override or copy the current nix file and update the version and hash - you get the version you want. It will use new version of the dependencies, but normally that's just fine.
Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by jljljl on Thursday May 25, 2023 @05:45PM (#36076659)
I thought this too, but we got a lot of requests for better version search + pinning. Adding overrides and looking up the right commit hash is a pretty cumbersome for a first time user.

For new projects though, I agree that using `latest` is generally the way to go.

Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by jljljl on Thursday May 25, 2023 @05:48PM (#36076703)
I do think Flakes help with the "throwing away old versions" issue. They also help with the monorepo issue by encouraging developers publish the Nix derivation in their own repo.

But they currently have a bit of a discoverability issue, and they need adoption by legacy packages to completely fix the problem.

Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by kaba0 on Friday May 26, 2023 @02:05AM (#36080593)
> just continually throw away old versions in nixpkgs head

It is simply completely infeasible to store and build every version of every package.

While it is not user friendly to find a package’s version X, it is either just overriding the version field and the hash, or referencing another version of the nixpkgs tree and building the same package from there.

It’s not trivial with other package managers either, and on top you can easily get into inconsistent states with those — nix can handle any version of any software correctly.

Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by jljljl on Friday May 26, 2023 @11:48AM (#36085718)
I don't think the answer is to have every version at the nixpkg head, for exactly the reasons you mentioned.

But I do think there's a search issue where mapping the version field of a package to the commit hash or nixpkgs version is not trivial or user-friendly, especially for multiple packages.

Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by peterbecich on Sunday May 28, 2023 @02:18AM (#36101641)
I believe the Hackage.nix library [1] has a solution to this problem, specifically for Haskell packages.

The `text` package [2] has many major, minor, and fix versions. Many of these versions are still required by projects. However, the current Nix Packages only has one version [3], and maybe a second version under a different name.

Hackage.nix will provide any version of `text` published on Hackage.

One downside is that `text` is not pulled from Nix Packages. The project's specific version of `text` must be cached elsewhere, or built from source. So it is slower.

[1] https://github.com/input-output-hk/hackage.nix [2] https://hackage.haskell.org/package/text [3] https://search.nixos.org/packages?channel=22.11&show=haskell...

Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by whateveracct on Thursday May 25, 2023 @11:34PM (#36079506)
On the flipside, it's easy to change the version of any package to any git rev and automatically rebuild any downstream dependency.
Re:Super Colliding Nix Stores: Nix Flakes for Millions of De... by chriswarbo on Friday May 26, 2023 @04:10AM (#36081416)
> Finding a specific version of a package is also really difficult. It requires you to find the commit hash in the Nixpkgs repo that last contained it, and there isn't a central search engine that lets you map version to commit.

I don't understand this: Nixpkgs can only ever contain a tiny proportion of package versions. It's meant as a mostly-consistent set: e.g. if you want packages "foo", "bar" and "baz", that should work.

It's not intended for "foo-1.2", "bar-0.2" and "baz-99"; let alone toggling the super-duper compiler flag and applying Alice's stability patches. However, the latter are intended by the functions which define those packages; and it's exactly why so many "override" functions are provided (to do all of the above, we usually just need '.overrideAttrs').

If you want a consistent set of packages, the usual approach is to run some sort of dependency solver (e.g. many of the 'foo2nix' tools), and map over its result. For example, at work our Scala projects run 'mvn2nix' during their build; use import-from-derivation to turn the results into a local Maven repository; and use that to build the project in Maven's "offline mode".