News for hackers. Stuff that matters.
Re:How Nix and NixOS get so close to perfect by KronisLV on Sunday April 24, 2022 @05:21AM (#31142446)
> I don’t really think this is a problem at all? I mean, at the particular design point the language occupies there are basically three possibilities:

How so? Where's the Java/Python-like languages in this consideration, why wouldn't those work? Or something more like Ansible?

I'm pretty sure that you can express the same concepts for the most part in something imperative as well.

Re:How Nix and NixOS get so close to perfect by cwp on Sunday April 24, 2022 @06:07AM (#31142659)
Yes and no. Yes, because Turing completeness. But no, because if you tried to do something like the Nix package manager with a language like Python or Java or even Scheme (yeah, I'm looking at you Guix), you'd end up with a worse system.

What's great about Nix the language is that its design matches beautifully with its purpose. Nixpkgs is a single giant program, the output of which is a set of 70,000 package definitions. Since Nix uses lazy evaluation, you can specify one of the attributes of that set, a single package, and the Nix interpreter will evaluate just that one package expression. Of course, if that package depends on other packages, those will get evaluated as well, because they're needed as inputs to the package you care about. So package dependencies emerge naturally as computational dependencies between function invocations. This is really powerful, and over the years, Nixpkgs has evolved more and more sophisticated ways of defining packages - package overrides, the overlay system, and most recently, flakes.

So yes, using an imperative language would technically be possible, but I doubt that you could use it to build the largest and freshest package repository with a fraction of the contributors that other packaging systems have.

Re:How Nix and NixOS get so close to perfect by tjoff on Sunday April 24, 2022 @07:37AM (#31143084)
I really struggle with selling argument of lazy evaluation in the nix context, it is not a big deal. Sure, nix has been built around it, but it could just as well have been built around something else. And most certainly can't see how that would influence the number of packages.

And honestly, the package situation is the biggest downfall of nix(os), there are tons of packages, yes. But they are often buggy (simply because nix is so different), not maintained (understandable, people move on), or plainly missing (because the few maintainers don't cover even popular niches). Of course that is going to be the case for a new system, but I'm flabbergasted every time it is being touted as an advantage of nix(os). And I feel that it is quite counter-productive tricking people into the ecosystem and giving the impression that it is on par or even better than any mainstream OS. It is making great progress, but it has a long way to go.

Re:How Nix and NixOS get so close to perfect by ris on Sunday April 24, 2022 @07:55AM (#31143201)
> And honestly, the package situation is the biggest downfall of nix(os), ... but it has a long way to go.

I have a big problem with "it has a long way to go" arguments. "It has a long way to go" in comparison to what? Every project I know of could be described as having "a long way to go".

If we're comparing it with ubuntu/debian, there are few packages that ubuntu/debian have that nixpkgs is missing. On the other hand, what's the status of getting kubernetes in ubuntu/debian? Last I heard it was mired in https://lwn.net/Articles/835599/. And what happens if you use the ubuntu/debian prometheus package? You currently get a version that many people would consider "too ancient to be useful". I could easily use these points to argue that debian has "a long way to go".

The real answer is "they're different". It's a mistake to think the two are just trying to replicate each other piece for piece.

The same goes for people arguing that the linux desktop has "a long way to go", whether they were making that argument in 2001 or 2021.

Re:How Nix and NixOS get so close to perfect by tjoff on Sunday April 24, 2022 @08:05AM (#31143266)
> I have a big problem with "it has a long way to go" arguments. "It has a long way to go" in comparison to what? Every project I know of could be described as having "a long way to go".

In comparison to any other mainstream OS nixos is going to have a lot of friction for most usecases. That is what I mean with it having long ways to go.

Part of that is configuration.

Other part of it is the package system being very immature. It is a direct consequence of the massive task nix has set out for itself, but that isn't always comforting for the end-user. And oh boy, the situation with looking for github-issues for packages is a freaking nightmare.

Just because nixos has a package for it doesn't mean that it does what it should or is up to date. That of course isn't the case for any package management system. But, all other mainstream OSes have matured. Nixos has a long way to go.

I love nixos, it is my main driver on my laptop and I run many VMs with it. If the future isn't incorporating the selling points of nixos I'm going to be dissapointed. But unless you want your OS to be your hobby I most certainly would not recommend nixos.

Nixops is another things that disappoint me / has long ways to go. It is the logical extension of what nixos is and it isn't really mature. There are lots of competing efforts for it and as someone willing to really invest time in it I just get exhausted.

Re:How Nix and NixOS get so close to perfect by sshine on Sunday April 24, 2022 @08:52AM (#31143480)
I run Ubuntu and MacOS because I don’t want my OS to be my hobby. I’ve run LFS, Slackware, Gentoo, FreeBSD, OpenBSD, Arch, Ubuntu Server (on laptop). I’ve spent a lot of time tweaking Xmonad, custom keyboard layouts, etc.

I would love for a distro with declarative configuration management. But I don’t have time for more hobbies right now.

Re:How Nix and NixOS get so close to perfect by emteycz on Sunday April 24, 2022 @06:16AM (#31142698)
Well said. Using imperative language would be possible but then it'd look like a bunch of function invocations, not like a class declaration. And people would be asking "this is crazy, why don't you create a DSL for this? My IDE can't ever understand this".
Re:How Nix and NixOS get so close to perfect by daptaq on Sunday April 24, 2022 @01:23PM (#31146115)
> even Scheme (yeah, I'm looking at you Guix), you'd end up with a worse system.

This is the first time I would have heard that point of view, could you elaborate?

Re:How Nix and NixOS get so close to perfect by deltaonefour on Sunday April 24, 2022 @11:38AM (#31145067)
It can be done without the language itself having lazy evaluation. Each package can just have a function that outputs some data structure and the lazy evaluation happens on the package builder.
Re:How Nix and NixOS get so close to perfect by soraminazuki on Sunday April 24, 2022 @04:34PM (#31148055)
The Nix language's main concern is handling large scale configuration. General purpose programming and scripting is completely out of scope.

The Nixpkgs repository has over 80,000 packages [1] contained in a single key-value store. The design of the Nix language not only makes this sustainable, but even compelling. For one thing, Nix package descriptions are composable and easy to manipulate programmatically. It's common practice to create new packages on the fly out of existing package definitions, and it's what the Nix language is particularly good at. Things like this add a lot of complex inter-dependencies within the Nixpkgs code base, and it still manages to be comprehensible.

The same approach isn't practical with general purpose imperative programming languages. A quick glance at Nixpkgs repository reveals that it contains over 2.4 million lines of Nix code. Imagine that much Java/Python code manipulating a single global mutable data structure. It can only result in a nightmare.

Additionally, the notion that the Nix is Haskell and is therefore incomprehensible is an often repeated meme without any backing details and needs to stop. If any language is similar to Nix, it's Jsonnet. But I haven't seen a single person criticizing Jsonnet [2] for being Haskell. If you can understand JSON and have no problem writing functions for any common language, then there really should be no problem learning the Nix language.

[1]: https://search.nixos.org/packages [2]: https://jsonnet.org

Re:How Nix and NixOS get so close to perfect by colordrops on Sunday April 24, 2022 @05:40AM (#31142529)
I don't see how you could maintain purity and reproduceability with something imperative. I used ansible for a while and it was a mess that broke half the time and was difficult to maintain.
Re:How Nix and NixOS get so close to perfect by tracnar on Sunday April 24, 2022 @07:13AM (#31142962)
Doesn't Bazel do exactly that with their Starlark language based on Python?

https://bazel.build/rules/language#mutability

Re:How Nix and NixOS get so close to perfect by pharmakom on Sunday April 24, 2022 @03:24PM (#31147362)
Key aspects of Bazel are not imperative. For example the “load” functions commands must not be computed.
Re:How Nix and NixOS get so close to perfect by deltaonefour on Sunday April 24, 2022 @11:48AM (#31145182)
You can. So long as the impurity is scoped to be local within a package. And the system itself only reads the output of your language to configure something. Example:

  def packageInstaller(pkgs: Pkg) -> Derivation:
      x = {
          req: [pkgs.a, pkgs.b]
          build: 
          ...
      }

      x[req].append(pkgs.c)
      return buildDerivation(x)
packageInstaller outputs a single package description, but you can also create something similar that easily outputs an entire configuration for the OS. You just need to keep mutation scoped within the language. The configuration that is outputted is immutable and whatever thing that's processing that configuration is not within user control.

Keep in mind that mutability and immutability share a certain isomorphism. Within the nix expression language you often do something similar to mutation just as any other functional language. For example:

    listVersion3 = listVersion1 ++ AdditionalStuff
It's just that in functional languages you're forced to keep versions of each mutation.
Re:How Nix and NixOS get so close to perfect by tjoff on Sunday April 24, 2022 @07:54AM (#31143195)
How does that have anything to do with it being imperative and not the fundamental design of the system?

How nix is architectured is exactly why it achieves purity and reproducibility, not the configuration language. It isn't even a goal of ansible in the same sense, per design.

The building blocks of nix is simple, of course it can be implemented in something imperative.

Re:How Nix and NixOS get so close to perfect by xena on Sunday April 24, 2022 @08:00AM (#31143236)
Think about it like this: with Ansible you start from some unknown state of the system and then iteratively do actions to hopefully get what you want. With Nix you know what the base state of the system is, and the configuration language lets you describe your desired end state. Then the computer figures out what to do in the middle.

The fundamental design of Ansible is that you do imperative steps to reach an end state. The fundamental design of NixOS is that you describe the end state and the compiler derives the intermediate steps.

Yes, normally things work out with Ansible, but the devil is in the details and can involve a lot of fighting configurations that are non-cromulent.

Re:How Nix and NixOS get so close to perfect by tjoff on Sunday April 24, 2022 @08:09AM (#31143296)
Yes - that was exactly my point. It has nothing to do with the configuration language. The end result of nix(os) could just as well be done from a bash script (not advisable, no).

And the problems with ansible would not be any better off if they had used something like the nix-language instead.

Re:How Nix and NixOS get so close to perfect by xena on Sunday April 24, 2022 @08:12AM (#31143310)
Fun fact: Nix is really a compiler from a weird Haskell like DSL to bash!
Re:How Nix and NixOS get so close to perfect by sciurus on Sunday April 24, 2022 @02:42PM (#31146920)
> The fundamental design of Ansible is that you do imperative steps to reach an end state.

No, the fundamental design of configuration management systems (Ansible, Puppet, Chef, Salt, CFEngine) is that they are declarative. You say "this software should be installed, this user should exist, these files should have these contents and permissions, etc" and they figure out the right steps to take on your OS to move from the current state to your desired end state.

You are right though that they can produce unexpected results since they do not describe the entire system from the ground up.

Re:How Nix and NixOS get so close to perfect by soraminazuki on Sunday April 24, 2022 @05:10PM (#31148377)
I deal with Chef at work, and I wish that was even remotely true. It's not a declarative system at all because it does not have the properties of being one. Namely the part about being able to specify the end state.

Chef doesn't allow you to specify the end state. Instead it allows you to specify a list of actions to be taken during deployment. The end result is affected by bunch of external inputs sent from the Chef server and every single bit of state the target server is able to observe. It is therefore impossible to reliably determine the end state of the target server before deployment. Our team has been burnt by this more than once.

Other similar systems are more or less the same. They have every hallmarks of being an imperative system.

Re:How Nix and NixOS get so close to perfect by colordrops on Sunday April 24, 2022 @03:18PM (#31147280)
That's fair, though ansible has ordered steps, so it's not entirely declarative.

You are right that imperative definition is not a problem for reproduceability, but it does make things harder to reason about since you have to mentally keep track of state between steps.

Re:How Nix and NixOS get so close to perfect by ris on Sunday April 24, 2022 @08:01AM (#31143247)
> Or something more like Ansible?

If you make it more like Ansible, you will end up with ... Ansible.

If you want to use Ansible you are perfectly welcome to, but as a Nix person the entire Ansible way looks like a completely wrongheaded approach to system management.

Re:How Nix and NixOS get so close to perfect by sshine on Sunday April 24, 2022 @08:59AM (#31143531)
If manual Linux server maintenance is like working with clay, Ansible is like a clay gun. It works great when you have to initiate work on a server you have limited control over; e.g. one that is provisioned from a cloud provider. And where I don’t need more than a few commands, and never need to replay them.

Ansible is “bottom-up” (you start in the middle of a server’s lifetime without knowing everything); many more cloudy solutions like Kubernetes assume that a bunch of infrastructure is already set up; container registries, build servers, and the control plane.

My first Ansible use-case was to install K3s on a dedicated machine where I could only choose the OS image from a few options like Debian and CentOS.

Re:How Nix and NixOS get so close to perfect by mananaysiempre on Sunday April 24, 2022 @09:12AM (#31143616)
You could make a package manager or distribution along the principles of Nix with any reasonable language, of course. Insisting on first-order data (i.e. not functions) exclusively would probably have the standard problems of trying to cram into a declarative configuration something that really wants to be a program (“YAML ops”, programs-as-SQL-tables disasters like Windows Installer, XML hells like Ant and XSLT, etc.); but once you go all the way to a general programming language, sure.

I would probably prefer Lua to Python because while Python can sort of do declarative EDSLs they invariably suck, and the expressive power and general style of the two is largely similar; Java has all those problems times a hundred so I expect you would end up staging the whole thing into a config processor in Java and a config in something else; but while the ergonomics may vary it’s certainly not impossible. Guix, Pacman with Aconfmgr, or the venerable GNU Stow are all this to some degree.

That is not what I was saying, though. The semantics of the Nix language (JSON-like data model, lazy, higher-order, dynamic types) are a vital part of the implementation approach taken by the Nix package manager, and once those semantics are fixed the syntax (so I argued) has to be broadly similar to what it is now. (You could force a Java- or Python-like syntax upon an ML- or Scheme-like language if you really wanted, but the result would just be miserable to program in; the F# people tried but mostly gave up.)

Is that approach the right one? I suspect so, but it is still to early in the life of the whole idea to be really sure. The idea that a dynamically-typed lazy (thus pure) higher-order language is a good middle ground between a completely declarative configuration and a completely imperative setup script (neither of which can usually stand alone) is a fresh and interesting one, and I’d very much like to see it explored. I’ll be the first to admit that the documentation on Nix internals is lacking, but once the requisite source spelunking is done it’s delightful how the whole thing comes together and how much flexibility being written in itself affords it. (Did you know that NixOS was originally a fun experimental addon to the whole thing? The packaging language accidentally ended up being powerful enough to make a usable and novel system configuration language.)

But the complaint seemed to be that the syntax (which is relatively superficial and does not require particularly deep argumentation) is Haskell-like, not that the implementation approach (which is crucial to the whole design and requires arguments on the scale of a research paper) is wrong. My point was that the latter pretty much implies the former.

Re:How Nix and NixOS get so close to perfect by pharmakom on Sunday April 24, 2022 @03:32PM (#31147454)
> the result would just be miserable to program in; the F# people tried but mostly gave up.

Huh?

Re:How Nix and NixOS get so close to perfect by yakak on Sunday April 24, 2022 @07:04AM (#31142917)
> Or something more like Ansible?

This seems to be the argument that comes up over and over with go, JavaScript, Ansible, etc. Sure one can take an imperative language and try to follow FP principles to emulate good things. Then one is comfortable with how it works while not really understanding FP and when it trashes the system by being implemented as a sequence of trashes of the state it is just a learning experience of user error.