News for hackers. Stuff that matters.
Re:Ask HN: Why does every package+module system become a Rub... by pastage on Monday January 30, 2023 @06:20AM (#34578348)
2) In Debian a binary package has almost always just been a cache, are we thinking about Debian packaging in completely different ways? Are you perhaps talking about the aspect of doing an "build world"[0] that bsd, Gentoo and now Nix are better at?

[0] Edit: rebuilding your dependencies if you need it and handling that seemlesly CAN be hard on Debian. Something that even Nix struggles with even if they are best in class by far. It is also completely different from the notion of compiling a program. I wonder what you consider the goal of a package system.

Re:Ask HN: Why does every package+module system become a Rub... by tazjin on Monday January 30, 2023 @08:13AM (#34579145)
> are we thinking about Debian packaging in completely different ways?

Quite likely! The whole concept of separately building a source/binary package, and then uploading/"deploying" that binary package, already violates the notion of being "just a cache" for me. There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix (i.e. guarantees about the artifact being repeatable, and being addressable by its inputs rather than a human-curated tag or version number).

> Something that even Nix struggles with

Nix only struggles with it to the degree that some of the higher-level abstractions in the existing package set (which are built on top of the fundamental model, not part of it) can be confusing/underdocumented, but conceptually this is a simple thing to do in Nix. Even practically at this point it is usually simple - unless you're dealing with Haskell or Javascript, of course.

> I wonder what you consider the goal of a package system

I want it to let me describe a desired state, and then make it so. That state is best represented as a graph, like a Merkle tree, of the instructions for the individual steps and a way to address them. An individual step is a transformation (i.e. some program) executed over some sources, yielding some result (likely in the filesystem). I want any distribution of binaries to be a result of using this addressing scheme, and looking up/copying an already built equivalent artifact for that thing. I want a text file containing the string "Hello HN" to be represented by the same abstraction that represents a specific .so file, the `emacs` package, the configuration of my entire system, the configuration of a cluster of systems and so on. I want this system to be programmable so I can work around the shortcomings that its original designers missed.

Nix (and Guix) do large parts of this already, and are conceptually (though not yet technically) capable of doing all of it. Technically, so is something like Bazel - but its complexity and maintenance requirements make that prohibitively expensive (basically only Google can use Bazel like that, and even they have delineated areas where they "give up" on wrapping things in Bazel).

Re:Ask HN: Why does every package+module system become a Rub... by Nursie on Monday January 30, 2023 @10:48AM (#34581226)
> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally"

apt-get --build source package_name

It's unlikely to be fully repeatable to the point of producing an identical binary, but it's also not some esoteric set of procedures.

Re:Ask HN: Why does every package+module system become a Rub... by goodpoint on Monday January 30, 2023 @12:53PM (#34583617)
> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix

It's been there for 15 or 20 years. Also Debian has been doing reproducible builds before Nix.

Re:Ask HN: Why does every package+module system become a Rub... by anderskaseorg on Monday January 30, 2023 @01:33PM (#34584258)
Even in the best case where Debian satisfies all the reproducibility goals it sets for itself (i.e. the ~96% at https://tests.reproducible-builds.org/debian/reproducible.ht... becomes 100%), each Debian package is still only a reproducible function of the current versions of its build dependencies at the time it was built. Upgrading those dependencies can change the build output, and this is still normal and expected in Debian; it’s the entire point of a binNMU.

In order to reproduce a given Debian package from scratch, one would need to track down long chains of historical versions of packages from sid. Due to circular dependencies, these chains probably snake back to the very first versions of Debian before the availability of snapshot.debian.org. It’s possible that human judgment could be used to break some chains at an earlier point by substituting different versions, but this goes outside the guarantees provided by Debian’s reproducibility.

Nix guarantees that every package is built with the specified versions of its dependencies, and so on recursively. Some more rebuilding is necessary to maintain this guarantee, but in exchange, it enables stronger notions of reproducibility that Debian can’t.

Re:Ask HN: Why does every package+module system become a Rub... by 0x0 on Monday January 30, 2023 @01:39PM (#34584378)
Well of course it snakes back, and likely beyond debian's inception, your initial c compiler has to come from somewhere.
Re:Ask HN: Why does every package+module system become a Rub... by anderskaseorg on Monday January 30, 2023 @01:44PM (#34584455)
Right. Nix bootstraps from a fixed collection of seed binaries, and everything in any given commit of nixpkgs is derived from the seed binaries and versions of software that exist in that commit of nixpkgs. It doesn’t solve the “trusting trust” problem with those seed binaries, but it does solve the “arbitrarily long chains” problem.

(The work at https://www.gnu.org/software/mes/ aims to minimize the needed binary seed to a human-auditable size, and should be applicable to Nix and Guix.)

Re:Ask HN: Why does every package+module system become a Rub... by pxc on Monday January 30, 2023 @02:34PM (#34585186)
* * *
Re:Ask HN: Why does every package+module system become a Rub... by tremon on Monday January 30, 2023 @05:42PM (#34587619)
With Nix or Guix, when a cached binary artifact for a given package is missing, you may not even notice, the fallback to building from source is so transparent.

I consider this to be one of the main drawbacks of these systems -- it assumes that every machine is build-capable. I do not want compilers installed on my production servers (nor would I want my RPi to attempt to rebuild half the world because the package server was down). I'm not sure how Nix handles this, but I'm pretty sure that's not possible with Guix.

Re:Ask HN: Why does every package+module system become a Rub... by pxc on Monday January 30, 2023 @07:16PM (#34588835)
> it assumes that every machine is build-capable

It doesn't, fortunately. You can tell Nix to rely exclusively on the binary cache, or configure it to have 0 build threads locally and use remote builders.

For a server deployment, many popular choices don't have the server build anything, and in some cases it may not even have any users who are allowed to invoke `nix-*` or anything.

Re:Ask HN: Why does every package+module system become a Rub... by tazjin on Tuesday January 31, 2023 @04:04AM (#34592938)
To be fair, the whole concept of manually installing individual packages on production servers (especially when they run NixOS!) is a complete anti-pattern, so that situation wouldn't even occur during normal operation.
Re:Ask HN: Why does every package+module system become a Rub... by goodpoint on Monday January 30, 2023 @07:17PM (#34588840)
Please stop assuming I don't know what Nix and Guix do.

> you'd understand why this is a silly point to raise

It isn't.

Re:Ask HN: Why does every package+module system become a Rub... by pjc50 on Monday January 30, 2023 @10:18AM (#34580720)
Debian packages can have pre/postinstall scripts that mutate the overall state of the system in arbitrary ways. This is both (a) necessary to make some thing work and (b) ruins the theoretical model.
Re:Ask HN: Why does every package+module system become a Rub... by Nursie on Monday January 30, 2023 @10:43AM (#34581138)
Those dirty practicalities breaking my perfect mental model again!

(This is a huge issue in our field in all sorts of ways, and a constant source of both pain and compromise)

Re:Ask HN: Why does every package+module system become a Rub... by pxc on Monday January 30, 2023 @02:45PM (#34585342)
Alternatively: that pesky lack of discipline obstructing my optimizations and features again.
Re:Ask HN: Why does every package+module system become a Rub... by marcosdumay on Monday January 30, 2023 @10:44AM (#34581146)
> This is ... necessary to make some thing work

Things work in NixOS without the need of any of that. So, no it is not necessary.

It's a need created purely from the other directives of the Debian organization.

Re:Ask HN: Why does every package+module system become a Rub... by hiljusti on Monday January 30, 2023 @12:36PM (#34583349)
Not necessary, but certainly it's easier to implement and get stuff done. It starts to break down and show its cracks the larger and larger it gets, though.

The difference between almost all Linux or BSD packaging systems and nixpkgs is like the difference between dynamic and compile-time-type-checked programming languages.

Re:Ask HN: Why does every package+module system become a Rub... by pxc on Monday January 30, 2023 @02:43PM (#34585324)
You can have a much more conventional package manager without pre- or post-install hooks. One ex-Debian developer created such a package manager as a research project, presumably to prove a point to other Debian people with the same head-in-the-sand attitude about the state of Debian's package tooling.

It doesn't require the same kind of patching as Nix does (although sometimes patching does help avoid requiring those scripts).

Check it out: https://distr1.org/