News for hackers. Stuff that matters.
Re:Ask HN: Why does every package+module system become a Rub... by tazjin on Monday January 30, 2023 @08:13AM (#34579145)
> are we thinking about Debian packaging in completely different ways?

Quite likely! The whole concept of separately building a source/binary package, and then uploading/"deploying" that binary package, already violates the notion of being "just a cache" for me. There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix (i.e. guarantees about the artifact being repeatable, and being addressable by its inputs rather than a human-curated tag or version number).

> Something that even Nix struggles with

Nix only struggles with it to the degree that some of the higher-level abstractions in the existing package set (which are built on top of the fundamental model, not part of it) can be confusing/underdocumented, but conceptually this is a simple thing to do in Nix. Even practically at this point it is usually simple - unless you're dealing with Haskell or Javascript, of course.

> I wonder what you consider the goal of a package system

I want it to let me describe a desired state, and then make it so. That state is best represented as a graph, like a Merkle tree, of the instructions for the individual steps and a way to address them. An individual step is a transformation (i.e. some program) executed over some sources, yielding some result (likely in the filesystem). I want any distribution of binaries to be a result of using this addressing scheme, and looking up/copying an already built equivalent artifact for that thing. I want a text file containing the string "Hello HN" to be represented by the same abstraction that represents a specific .so file, the `emacs` package, the configuration of my entire system, the configuration of a cluster of systems and so on. I want this system to be programmable so I can work around the shortcomings that its original designers missed.

Nix (and Guix) do large parts of this already, and are conceptually (though not yet technically) capable of doing all of it. Technically, so is something like Bazel - but its complexity and maintenance requirements make that prohibitively expensive (basically only Google can use Bazel like that, and even they have delineated areas where they "give up" on wrapping things in Bazel).

Re:Ask HN: Why does every package+module system become a Rub... by Nursie on Monday January 30, 2023 @10:48AM (#34581226)
> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally"

apt-get --build source package_name

It's unlikely to be fully repeatable to the point of producing an identical binary, but it's also not some esoteric set of procedures.

Re:Ask HN: Why does every package+module system become a Rub... by goodpoint on Monday January 30, 2023 @12:53PM (#34583617)
> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix

It's been there for 15 or 20 years. Also Debian has been doing reproducible builds before Nix.

Re:Ask HN: Why does every package+module system become a Rub... by anderskaseorg on Monday January 30, 2023 @01:33PM (#34584258)
Even in the best case where Debian satisfies all the reproducibility goals it sets for itself (i.e. the ~96% at https://tests.reproducible-builds.org/debian/reproducible.ht... becomes 100%), each Debian package is still only a reproducible function of the current versions of its build dependencies at the time it was built. Upgrading those dependencies can change the build output, and this is still normal and expected in Debian; it’s the entire point of a binNMU.

In order to reproduce a given Debian package from scratch, one would need to track down long chains of historical versions of packages from sid. Due to circular dependencies, these chains probably snake back to the very first versions of Debian before the availability of snapshot.debian.org. It’s possible that human judgment could be used to break some chains at an earlier point by substituting different versions, but this goes outside the guarantees provided by Debian’s reproducibility.

Nix guarantees that every package is built with the specified versions of its dependencies, and so on recursively. Some more rebuilding is necessary to maintain this guarantee, but in exchange, it enables stronger notions of reproducibility that Debian can’t.

Re:Ask HN: Why does every package+module system become a Rub... by 0x0 on Monday January 30, 2023 @01:39PM (#34584378)
Well of course it snakes back, and likely beyond debian's inception, your initial c compiler has to come from somewhere.
Re:Ask HN: Why does every package+module system become a Rub... by anderskaseorg on Monday January 30, 2023 @01:44PM (#34584455)
Right. Nix bootstraps from a fixed collection of seed binaries, and everything in any given commit of nixpkgs is derived from the seed binaries and versions of software that exist in that commit of nixpkgs. It doesn’t solve the “trusting trust” problem with those seed binaries, but it does solve the “arbitrarily long chains” problem.

(The work at https://www.gnu.org/software/mes/ aims to minimize the needed binary seed to a human-auditable size, and should be applicable to Nix and Guix.)

Re:Ask HN: Why does every package+module system become a Rub... by pxc on Monday January 30, 2023 @02:34PM (#34585186)
* * *
Re:Ask HN: Why does every package+module system become a Rub... by tremon on Monday January 30, 2023 @05:42PM (#34587619)
With Nix or Guix, when a cached binary artifact for a given package is missing, you may not even notice, the fallback to building from source is so transparent.

I consider this to be one of the main drawbacks of these systems -- it assumes that every machine is build-capable. I do not want compilers installed on my production servers (nor would I want my RPi to attempt to rebuild half the world because the package server was down). I'm not sure how Nix handles this, but I'm pretty sure that's not possible with Guix.

Re:Ask HN: Why does every package+module system become a Rub... by pxc on Monday January 30, 2023 @07:16PM (#34588835)
> it assumes that every machine is build-capable

It doesn't, fortunately. You can tell Nix to rely exclusively on the binary cache, or configure it to have 0 build threads locally and use remote builders.

For a server deployment, many popular choices don't have the server build anything, and in some cases it may not even have any users who are allowed to invoke `nix-*` or anything.

Re:Ask HN: Why does every package+module system become a Rub... by tazjin on Tuesday January 31, 2023 @04:04AM (#34592938)
To be fair, the whole concept of manually installing individual packages on production servers (especially when they run NixOS!) is a complete anti-pattern, so that situation wouldn't even occur during normal operation.
Re:Ask HN: Why does every package+module system become a Rub... by goodpoint on Monday January 30, 2023 @07:17PM (#34588840)
Please stop assuming I don't know what Nix and Guix do.

> you'd understand why this is a silly point to raise

It isn't.