News for hackers. Stuff that matters.
Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by throwaway984393 on Tuesday September 14, 2021 @06:58PM (#28532661)
It's a nice idea but it won't gain traction as it's still designed with 2000's era computing in mind. Nobody manages an entire complex system within a specific Linux distribution anymore. There is a vast ecosystem of tooling, systems, protocols, networks, services, that comprise modern distributed systems. All these things exist regardless of the Linux distro.

In fact, I will predict right now the death of Linux as a dominant backend computing platform. It's sort of happened already. Sure, Linux is the kernel that runs the host machines that the cloud runs on. But the services are increasingly serverless/kernel-less, or run on micro-vms running either Linux or another kernel, or run in containers made of a half dozen different base distros. All the Linux I/O is just passed to an outer layer with a universal adapter, so you can mix and match networking, filesystems, logging platforms, policy governance, orchestration, scheduling, etc. I spend nearly all of my time building systems by tuning software and services that have almost nothing to do with Linux.

Linux distros will still be around in one form or another. But long gone are the days where most people define their system or security based on a particular distro. Hell, most of the software used in the cloud today isn't even packaged by distros. Linux has become just a kernel again, the distros just a fancy installer for GNU tools.

They're using a lot of advanced tech and advertising having to "maintain fewer virtual computers", whatever that means. But it also seems their tech is more complex and less compatible with existing components. There's no clear idea of how a developer's supposed to take their code on their laptop, test it, and ship it to production as one immutable image. If it's not as simple as containers, it's not going to replace them.

Also, it's interesting that they seem to allude to immutability, but don't mention the principle by name at all. It seems like whomever is developing this doesn't run large distributed systems.

Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by SkyMarshal on Tuesday September 14, 2021 @07:12PM (#28532774)
> Interestingly, of all the buzzwords this distro uses, none include "immutable", which is the single most important concept in modern systems.

Since this system is based on NixOS and the Nix package manager, immutable is implied. Any change you make to a Nix/NixOS system results in a full rebuild of the system, and pointers to the new system updated. Current and all prior builds remain immutable.

Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by throwaway984393 on Tuesday September 14, 2021 @07:14PM (#28532804)
Yeah I've never used Nix so I don't know how it relates to cloud systems. As a developer, how would I use Nix to develop an app on my machine, and ship it to 1,000 cloud-orchestrated systems, using the same immutable image? Is there any reason why one would use Nix with Docker instead of Debian with Docker?

Keep in mind that all large-scale systems work based on deploying container images. If it's not in a container image, it's not getting deployed.

Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by sterlind on Tuesday September 14, 2021 @07:26PM (#28532930)
Nix basically lets you compile to Docker containers, rather than running a pipeline of scripts with Dockerfiles. You can do that really fast, since you don't have to run the container to build it, and you can make images really small, because they'll contain exactly what they need.

It's like the difference between shipping an app and shipping an OS with the app-preinstalled, except in this case neither has any external dependencies to worry about.

Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by fwip on Tuesday September 14, 2021 @07:47PM (#28533143)
Also, the nix expression you build an image with is deterministic, unlike Dockerfiles, which by-design aren't.

(Dockerfiles are not the only way to build images, of course, but they are a common one).

Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by throwaway984393 on Tuesday September 14, 2021 @08:55PM (#28533725)
Well that's good that it can make container images. Sounds like the main advantage is it's faster to install packages in the container image. But I don't understand how this could prevent you from still needing a pipeline of scripts.

Installing packages is only a small part of building a typical container image, especially for custom apps being built to run in the cloud. Even if you have your dependencies, you need to compile, run tests, and potentially customize the files inside the container, typically using the programs already installed in the container.

So since you're already running a pipeline of scripts in a container right after installing package deps, I'm not sure why one would use Nix for this when Debian works just as well?

Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by sterlind on Wednesday September 15, 2021 @12:30AM (#28535285)
Nix isn't an OS, Nix is a build system, sort of like Bazel. The way you use Nix as an OS (NixOS) is to make Nix build an OS for you. Every time you install a package or whatever, that's just an incremental recompilation.

So say that you need to make a Docker container running nginx hosting a static website that you build with Jekyll. At a high level you'd write a Nix file like (very much pseudocode):

with let site = buildWithJekyll src={"./src";}; let mynginx = nginx { wwwroot = site; }; docker.buildLayeredImage { inputs: mynginx, entrypoint: "./nginx" }

Nix will run Jekyll to build the site, hash the build output and "site" gets that path. Then Nix takes Jekyll and writes a configuration file that sets wwwroot to the Jekyll output. Then Nix builds a container that runs nginx when it starts. If you just change the Nginx config it won't rerun Jekyll because it knows Jekyll's build inputs haven't changed.

Nix is a build system that runs other build systems. Comparing Nix to Debian doesn't really make sense, it's like comparing build pipelines to makefiles.

Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by aidenn0 on Wednesday September 15, 2021 @12:20AM (#28535247)
> Sounds like the main advantage is it's faster to install packages in the container image.

I think the main advantage is the builds are actually deterministic. Any Dockerfile with the equivalent of

   apt-get update
in it becomes non-deterministic.
Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by lom on Wednesday September 15, 2021 @12:10AM (#28535185)
I was building a docker image today with nix, but to my demise it ended up being over 90mb (loaded into the docker daemon). How can I make a docker image smaller?
Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by sterlind on Wednesday September 15, 2021 @12:31AM (#28535288)
Make sure you're not bringing in superfluous dependencies. Use nativeBuildInputs (like devDependencies) when appropriate rather than buildInputs (like dependencies).
Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by lom on Wednesday September 15, 2021 @03:24PM (#28543235)
Can I override stdenv to not pull in GNU stuff, but instead busybox. Or none at all? Stuff like that makes the image really small…
Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by sterlind on Thursday September 16, 2021 @12:40AM (#28548097)
stdenv shouldn't end up in the container at all. the container should literally just contain your derivation and its transitive dependencies. your app, its actual dependencies, and nothing else. it won't even contain bash unless you or something you reference adds it. stdenv stuff is just used to build, it doesn't end up in there. maybe use one of the nix tools to show the actual dependencies you're bringing in and cut the fat.
Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by SkyMarshal on Tuesday September 14, 2021 @09:27PM (#28534032)
I'm still relatively new to NixOS, having switched all my personal systems over to it this spring/summer. I don't do cloud dev atm, so haven't explored that use case yet.

But I believe NixOPs is the canonical way to do what you're describing in production/at scale:

https://github.com/NixOS/nixops

https://nixos.org/nixops/manual/

(more below)

If you want a more experienced answer, I suggest asking on the NixOS forum or subreddit, people are quick to answer in both places.

https://discourse.nixos.org/

https://www.reddit.com/r/NixOS/

----

More useful NixOPs guides

https://www.thedroneely.com/posts/nixops-towards-the-final-f...

https://ops.functionalalgebra.com/nixops-by-example/

https://nixops.readthedocs.io/en/latest/

Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by Ericson2314 on Tuesday September 14, 2021 @09:14PM (#28533908)
You basically wrote "it won't succeed because everything is going to go to shit". Well, maybe, but I hope not!

Our industry shamelessly imitates of the limping, stagnant, service economy in doing these things, and we call ourselves the future!

Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by seniorivn on Tuesday September 14, 2021 @07:11PM (#28532766)
you missed the fact that it's nixos based. It means it is a system to declaratively define compartmentalized well integrated(including GUI) system, at the moment it's based on Linux and other regular Gnu/Linux but it could relatively easy change any of them
Re:Spectrum OS: a declarative, reproducible, compartmentaliz... by throwaway510921 on Wednesday September 15, 2021 @06:19AM (#28537363)
>If it's not as simple as containers, it's not going to replace them.

Nix is more (or less) than a container. It depends, what you are targeting with your nix-project. You can either build a minimal NixOS, that has only a kernel, the minimum required user land and your project and its dependencies.

Or build a stand alone docker (etc.) image and spin it up.

Or you can use the nix shell to replicate just the dependencies of your project in that sub shell for other developers or testing.

I just recently saw a convincing video about nix, comparing it to docker.

https://www.youtube.com/watch?v=pfIDYQ36X0k