News for hackers. Stuff that matters.
Re:Will Nix Overtake Docker? by exdsq on Monday November 29, 2021 @10:13PM (#29388505)
Oh god I hope not. Having worked in > 100kloc nix environments I am completely turned off of the idea. I really really tried, I installed NixOS as my main OS and used Nix whenever I could to try and pick it up, but it's such a complex beast I felt it slowed everything down. Simple tasks that would take 10 minutes in Docker suddenly became DevOps tickets. I suddenly had to write bindings for tools rather than apt-get them in a Dockerfile. Build times in Haskell were bad enough but suddenly you had nix into the mix and I'd be wasting half a day as Nix built the entire world - solved with internal caching servers but have fun trying to work out how to push something to that correctly.

There's a blog post that goes around from time to time about how a company have three risk tokens to allocate per project on non-boring technologies. Nix seriously needs all three. It's the only technology I've ever vetoed at a startup because I've seen the hell hole it can become - perhaps not for the DevOps engineers who know it, but the team who have to suddenly work within the resulting environment.

Re:Will Nix Overtake Docker? by rgoulter on Monday November 29, 2021 @10:43PM (#29388700)
> There's a blog post that goes around from time to time about how a company have three risk tokens to allocate per project on non-boring technologies.

https://mcfunley.com/choose-boring-technology

Re:Will Nix Overtake Docker? by exdsq on Monday November 29, 2021 @10:45PM (#29388714)
Thanks, this is the one! Highly recommend it, especially when you're in the HN echo chamber of new and interesting technologies
Re:Will Nix Overtake Docker? by amarshall on Tuesday November 30, 2021 @02:01AM (#29389692)
Worth noting in this context of OP, though, is that Nix predates Docker by about a decade.
Re:Will Nix Overtake Docker? by eximius on Tuesday November 30, 2021 @02:14AM (#29389751)
That is extremely concerning, then, that it is so difficult to use after all that time.

I had hopes in the back of my head that "maybe it'll get better/more ergonomic in the next few years".

Re:Will Nix Overtake Docker? by beermonster on Tuesday November 30, 2021 @02:23AM (#29389796)
Is it? Or is it just a tool that requires a different mindset so those coming to it from another regular distro? If the latter, then I’m not sure it’s something they will see as a problem but more its very ethos.
Re:Will Nix Overtake Docker? by rgoulter on Tuesday November 30, 2021 @02:33AM (#29389833)
Maybe an apt comparison is git.

It's just as easy to make a mistake with git today as it was however many years ago; git hasn't fundamentally changed in ways that make it easier. Git still more/less requires you to have a good understanding of what's going on in order to be comfortable using it.

But, since use of git is now widespread, it's less of an issue. And the git CLI has seen some UX improvements.

Nix is very weird. I'm sure there are some ways its UX could be nicer; but to some extent, doing things in a strict/pure way is always going to have friction if the tools/environments don't easily support that.

Re:Will Nix Overtake Docker? by eitland on Tuesday November 30, 2021 @03:04AM (#29389979)
> It's just as easy to make a mistake with git today as it was however many years ago;

Pro tip from an actual pro :

git became significantly easier for me once I decided

1. to just use a good comfortable gui (at least one tui also look good) for anything beyond commit and push. (maybe not everything can be done from your gui of choice but at least you get a good overview of the situation before you dive in with the cli.)

2. and to just stick to the basics. (And when you need to help someone else who didn't, take it carefully even if that means you don't look like a wizard.)

In fact even working on the command line feels easier once I had worked in a good gui for a while.

Don't believe experts who claim that you need to study vim and stick to the git cli to "get it".

But of course: if cli is your thing, use it, just stop writing blog posts that claiming it is the only true way.

Re:Will Nix Overtake Docker? by carlmr on Tuesday November 30, 2021 @09:54AM (#29392739)
Which git GUI do you recommend? The one in VSCode I find even more confusing than the CLI.

I do agree with you that some workflows are just easier with a GUI, since I used to use TortoiseSVN and it was much nicer for diffing two commits than the CLI is. But I haven't really dug into git GUIs.

Re:Will Nix Overtake Docker? by copperx on Tuesday November 30, 2021 @10:33AM (#29393206)
Magit is the most recommended one. But it comes with an Emacs dependency.
Re:Will Nix Overtake Docker? by eitland on Tuesday November 30, 2021 @02:20PM (#29396270)
I recommend finding one or more that you like, my preferences doesn't always match everyone elses ;-)

At times I have used ungit or git extensions (which, despite its name contains a full desktop application.

Today I use VS Code with a combination of Git Branch for visualization and Git Lens for the rest. Git Lens might need some configuration (I rarely use all the panes, but I use the interactive rebase tool from it as well as some annotationn features. For the visual rebase tool to work I had to configure VS Code as my git editor, but that is how I like it anyway.)

Again: try a few, find one that makes sense for you.

Jetbrains tools for example are generally good, liked by many and well though I understand, but still the git visualization (and a couple of other things) consistently manages to confuse me.

Re:Will Nix Overtake Docker? by rgoulter on Tuesday November 30, 2021 @08:16PM (#29399529)
I found this website useful for helping me understand what was going on with git. https://learngitbranching.js.org/

GUI vs CLI shouldn't be about which is a less confusing way of using git.

(Though, yes, magit is excellent, and there are very few tools which come close).

Re:Will Nix Overtake Docker? by McP on Tuesday November 30, 2021 @03:24PM (#29397016)
https://www.syntevo.com/smartgit/ It's the most similar I've found to TortoiseHg which is the standard, and excellent, Mercurial GUI. (TortoiseGit does not live up to the family name)
Re:Will Nix Overtake Docker? by peanball on Wednesday December 01, 2021 @01:09AM (#29401184)
Fork, hands down. https://fork.dev
Re:Will Nix Overtake Docker? by KronisLV on Tuesday November 30, 2021 @10:38AM (#29393265)
Here are some recommendations:

SourceTree: https://www.sourcetreeapp.com/

Windows and Mac. Free. Feels sluggish, but is also really dependable, the graph view is lovely and it covers most of the common things that you want to do - also, staging/discarding chunks or even individual lines of code is lovely. Oh, and the Git LFS integration, and creating patches is also really easy. And it gives you the underlying Git commands it uses, in case you care about that.

GitKraken: https://www.gitkraken.com/

Windows, Mac and Linux. May need commercial license. Feels like a step up from SourceTree, but i find that using this for commercial needs is a no go. If that's not an issue, however, it has a good UI, is nice to work with and just generally doesn't have anything i'd object to. IIRC it saved my hide years back by letting me do a ctrl+z for a repo after accidentally forcing to the wrong remote, so that i could fix what i had done (memory might fail me, was years ago), just generally feels intuitive like that.

Git Cola: https://git-cola.github.io/

Windows, Mac and Linux. Free and open source. Perhaps one of the more basic interfaces, but as far as free software goes, it does what it sets out to do, and does it well. I use this on Linux, whenever i want to have that visual feedback about the state of the repo/staging area or just don't feel like using the CLI.

TortoiseGit: https://tortoisegit.org/

Windows only. Free. Recommending this just because you mentioned TortoiseSVN. If you just want a similar workflow, this is perhaps your best option. Honestly, there is definitely some merit to having a nice file system integration, i rather enjoyed that with SVN.

Whatever your IDE has built in: look at your IDE

On any platform that your IDE runs on. Same licensing as your IDE. Some people just shop around for an IDE that they enjoy and then just use whatever VCS workflows that they provide. I'd say that VS Code with some plugins is really nice, though others swear by JetBrains' IDEs, whereas others are fine with even just NetBeans or Eclipse (Java example, you can replace that with Visual Studio or whatever). If you're working within a particular stack/IDE, that's not too bad of an idea.

The CLI: https://git-scm.com/

Windows, Mac and Linux. Free and open source. You'll probably want to know a bit of the CLI anyways, just in case. Personally, i'm still way too used to using a GUI since dealing with branches and change sets just feels like something that's more easy when visual, but the CLI has occasionally helped me out nonetheless.

Actually, here's a useful list of some of the Git GUIs: https://git-scm.com/downloads/guis

For example, did you know that there are some simple GUI tools already built in, git-gui and gitk?

Re:Will Nix Overtake Docker? by pjmlp on Tuesday November 30, 2021 @07:09AM (#29391266)
I avoid most git usability issues by using it as SVN.

When something goes wrong I just bork the whole repo and clone it again, then manually merge the last set of saved changes.

Re:Will Nix Overtake Docker? by carlmr on Tuesday November 30, 2021 @09:55AM (#29392756)
Since learning about reset --hard, I need about 80% less of that.
Re:Will Nix Overtake Docker? by clem on Tuesday November 30, 2021 @11:44AM (#29394102)
"git reflog" has also been a blessing in getting out of scrapes.
Re:Will Nix Overtake Docker? by tomcam on Tuesday November 30, 2021 @07:38AM (#29391476)
Thank you! Me too. I can finally come out of the closet.
Re:Will Nix Overtake Docker? by pjmlp on Tuesday November 30, 2021 @08:57AM (#29392109)
I has been a long journey since I started with RCS back in 1998, but given that git now is everywhere, the least I deal with its complexity the better.

In some bad days I even miss Clearcase.

Re:Will Nix Overtake Docker? by 112233 on Tuesday November 30, 2021 @11:13AM (#29393715)
> It's just as easy to make a mistake with git today as it was however many years ago; git hasn't fundamentally changed in ways that make it easier.

Easier? Who needs that? Soon then you will have common unlearned folk and peasants trying to use git. </snark>

They had the super obtuse git reset, that was four different things bolted together, so they fixed it by adding git restore, that does slightly different things, but you still need both...

Re:Will Nix Overtake Docker? by amarshall on Tuesday November 30, 2021 @02:41AM (#29389861)
The latter. As a (bad) analogy: it’s a bit like learning Haskell. If you’ve only ever worked in strictly-evaluated, imperative languages, it’s not gonna be easy to learn. Nevertheless, Haskell predates e.g. Go by two decades.

Nix (and Haskell) has its warts, and undoubtedly a new system would avoid them (compatibility needs make some changes very challenging), but the fundamental difficulty remains because it is fundamentally different and solving a truly difficult problem set.

Re:Will Nix Overtake Docker? by KronisLV on Tuesday November 30, 2021 @10:21AM (#29393056)
> That is extremely concerning, then, that it is so difficult to use after all that time.

> I had hopes in the back of my head that "maybe it'll get better/more ergonomic in the next few years".

I feel much the same way about wanting to run something like FreeBSD and having it just work, as opposed to running into weirdness because of the driver situation, with which GNU/Linux seems to be getting better at (even though you sometimes are forced to install proprietary ones for optimal experience, should get better in the next decade).

So, might have to wait for a bunch more years, or just pick something else, like OpenBSD, or just run in a set of constraints for having a really predictable and well supported hardware configuration, which isn't always possible. Alas, my off brand Polish netbook will just have to wait before i can (hopefully) run BSD on it comfortably. Well, short of making the missing pieces of puzzle myself, for which i'd probably also need a decade or so of systems level development experience, as opposed to just web dev and occasionally dabbling in lower level stuff to mediocre success. Of course, by then the netbook itself might not be relevant, so who knows.

I also feel much the same way about GNU/Hurd - a project that is conceptually interesting, but isn't yet quite stable, even though apparently you can get Debian for it: https://www.gnu.org/software/hurd/

Now, i don't have almost any experience with it, apart from reading about it, but some people tried figuring out when it could be released based on the bug reports and how quickly they were being addressed, and the number that they came up with was around 2070 or so.

In summary, there are probably projects out there, for which their age isn't necessarily related to how usable they are, whereas other pieces of software will have never truly achieved that stability in the first place. Not all old software is good (edit: at least not for certain use cases).

Of course, there are exceptions, for example, you can look at Apache2/httpd: it is regarded as old and dated for the most part, however just recently an update was released, which added mod_md. It now lets you get Let's Encrypt certificates without external tools (like Certbot), in some ways setting it ahead of Nginx: https://httpd.apache.org/docs/2.4/mod/mod_md.html Not all old software is always boring.

Same for Docker and any other tools. If developers use tool A for use case X, instead of tool B, then maybe there are some very good reasons for this widespread usage? That's also probably the answer to this debate - regardless of their conceptual/technical benefits, the usability will probably decide which tool will win in the long term.

Re:Will Nix Overtake Docker? by lproven on Wednesday December 01, 2021 @09:00AM (#29403648)
My suspicion is that the HURD folk seized on new shiny that was not finished enough, and it's dogged the project ever since. Periodically the already-inadequate number of people have got diverted by investigating L4, then CoyotOS, then Viengoos.

I also submit that to understand why that happened, one needs to consider the context at the time that the decision was made:

http://www.h-online.com/open/features/GNU-HURD-Altered-visio...

http://www.groklaw.net/article.php?story=20050727225542530

« RMS was a very strong believer -- wrongly, I think -- in a very greedy-algorithm approach to code reuse issues. My first choice was to take the BSD 4.4-Lite release and make a kernel. I knew the code, I knew how to do it. It is now perfectly obvious to me that this would have succeeded splendidly and the world would be a very different place today.

RMS wanted to work together with people from Berkeley on such an effort. Some of them were interested, but some seem to have been deliberately dragging their feet: and the reason now seems to be that they had the goal of spinning off BSDI. A GNU based on 4.4-Lite would undercut BSDI.

So RMS said to himself, "Mach is a working kernel, 4.4-Lite is only partial, we will go with Mach." It was a decision which I strongly opposed. But ultimately it was not my decision to make, and I made the best go I could at working with Mach and doing something new from that standpoint.

This was all way before Linux; we're talking 1991 or so. » -- Friar Thomas Bushnell

They looked at BSD. The BSD people were not sure, so RMS decided on another path.

If GNU had picked the BSD kernel, there could have been a working Free xNix before Linux and things would have been very different.

Secondly, I think it's important when discussing microkernels and microkernel OSes to consider more than the most famous one: Mac OS X.

Xnu is based on Mach, but it's not a pure microkernel: its Xnu kernel contains a large, in-kernel "Unix server" derived from BSD code. This was done for performance reasons – remember, macOS is Mac OS X is NeXTstep, written for a 25MHz 68040 in around 1987-1988.

There are better examples. QNX is probably the best: a working, true-microkernel, Unix-like OS, dating from 1982. At one point shared-source but not any longer.

There is also Minix 3, which is a different OS from Minix 1 & 2, the OS that inspired Linux and upon which Linux was initially bootstrapped.

Minix 3 is still quite limited: no SMP, some missing APIs etc. But QNX proves that a true microkernel, on generic hardware such as x86 and ARM, can support SMP and so on.

Re:Will Nix Overtake Docker? by exdsq on Tuesday November 30, 2021 @08:02AM (#29391640)
Huh, cool! This is like Python being older than Java :mindblown:. Still, old technologies can be just as risky as new ones.
Re:Will Nix Overtake Docker? by roguas on Thursday December 02, 2021 @09:58AM (#29416925)
Nix popularity is still lower than Docker, even during dotcloud era(I think this was the og company). Boring means common, unnuanced, rather than older or first.
Re:Will Nix Overtake Docker? by gurjeet on Tuesday November 30, 2021 @01:17AM (#29389483)
Re:Will Nix Overtake Docker? by Cthulhu_ on Tuesday November 30, 2021 @04:32AM (#29390443)
I wish I had seen that (and fully understood the implications) years ago, I mean it's been in the back of my head all the time but never explained this well. I've referenced it a couple of times now, especially the slides showing the graphs of technologies and links which is a really succinct way of saying "adding one technology multiplies complexity".
Re:Will Nix Overtake Docker? by laumars on Tuesday November 30, 2021 @02:33AM (#29389836)
I hadn’t seen that before but it perfectly sums up my approach to tech. I’m going to share this the next time someone questions why I take a pragmatic approach rather than jumping on the next new tech stack

Thanks for sharing

Re:Will Nix Overtake Docker? by kwertyoowiyop on Tuesday November 30, 2021 @10:25AM (#29393096)
Strangely, once a team adds one risk, they feel more comfortable adding a second risk. Ditto for the third risk, etc. The psychology is weird.
Re:Will Nix Overtake Docker? by chriswarbo on Tuesday November 30, 2021 @05:02AM (#29390585)
> apt-get them in a Dockerfile

> Nix built the entire world

You're comparing apples to oranges.

Docker just runs commands. If you want to run commands, you can do that in Nix using e.g. nixpkgs.runCommand.

apt-get just fetches prebuilt binaries. If you want to fetch prebuilt binaries, you can do that in Nix using e.g. nixpkgs.fetchurl.

You can even use runCommand to run apt-get (I do this to use Debian's Chromium package on 32bit NixOS)

In contrast, it sounds like you were using Nix as a build tool, to define your entire OS from scratch (presumably using Nixpkgs/NixOS). In which case apt-get isn't a comparable command; instead, the Debian equivalent would be:

    apt-build world
Guess what that does?
Re:Will Nix Overtake Docker? by taeric on Monday November 29, 2021 @10:21PM (#29388557)
Funny, because I feel that simple tasks that would take minutes in my machine are now a dev adventure with docker.

And I mean funny. I suspect it is different mindsets. And I personally like that both seem to be thriving.

Re:Will Nix Overtake Docker? by aspaceman on Monday November 29, 2021 @10:49PM (#29388739)
I feel similarly with Docker. But it's easier to explain to newer folks because it's only a single layer of abstraction above shell commands.
Re:Will Nix Overtake Docker? by aequitas on Tuesday November 30, 2021 @03:27AM (#29390097)
The issue with simple solutions is that the underlying problem still has the same complexity. This complexity rises up sooner or later and depending on your abstraction will be nice caught within it or make a complete mess.
Re:Will Nix Overtake Docker? by dnautics on Tuesday November 30, 2021 @01:12AM (#29389472)
I really wish something like singularity containers had taken over -- that was literally just shell commands.
Re:Will Nix Overtake Docker? by mst on Tuesday November 30, 2021 @06:15AM (#29390963)
Possibly https://buildah.io/ would be of interest.
Re:Will Nix Overtake Docker? by dnautics on Wednesday December 01, 2021 @12:15PM (#29406310)
I do use podman, not really stoked about buildah because it's still a bunch of buildah commands in a shell-driven DSL, which I suppose is better than a dockerfile. Singularity was literally a build file that was a shell script divided into sections that denoted phases.
Re:Will Nix Overtake Docker? by mst on Monday December 06, 2021 @09:44AM (#29460274)
Bit late now but you could probably make something like it using buildkit - https://github.com/moby/buildkit/blob/master/examples/buildk...
Re:Will Nix Overtake Docker? by jokethrowaway on Tuesday November 30, 2021 @09:59AM (#29392801)
I can't say Docker is simpler than Nix.

I found Nix way easier, but the documentation is very... concise.

Re:Will Nix Overtake Docker? by taeric on Tuesday November 30, 2021 @01:17AM (#29389485)
I call this. It isn't a single abstraction over shell. It is a single closure over shell. And that is surprisingly hard to explain.

Again, though, I am glad both are thriving. Competition of implementation is... not that compelling to me. Competition of approach, is amazing.

Re:Will Nix Overtake Docker? by goodpoint on Tuesday November 30, 2021 @07:27AM (#29391380)
It's well documented that Docker and especially docker hub had a terrible impact on security.

Once you factor in the efforts required in the long term to mitigate a decade of bundling gigabytes of applications and libraries it's a huge "dev adventure"

Re:Will Nix Overtake Docker? by Frost1x on Tuesday November 30, 2021 @08:31AM (#29391870)
More like developer Towers of Babel waiting to collapse on every build attempt. Docker is fine if it's used correctly but it does add a layer of complexity, it doesn't abstract it away..

The way I see it rampantly (ab)used is as a shortcut to get some software up and running by leveraging a public image and passing tech debt for some component of one's system onto the maintainer of the Docker image, then cobbling together multistage builds from those and microservice architectures to try and support this tech debt model. Sometimes it works, many times it breaks, often it turns into complete and utter nightmares to deal with.

"Amazing Andy got this fantastically complex system up and running in a week all by their self way under time and budget and it works. Now I just want to add this feature or modify this one thing and you're telling me that's going to take how long?" Yea, I've seen this more times than I care to admit.

Re:Will Nix Overtake Docker? by kwertyoowiyop on Tuesday November 30, 2021 @10:26AM (#29393108)
“You mean Amazing Architect Andy who just left for Google?” :-(
Re:Will Nix Overtake Docker? by KronisLV on Tuesday November 30, 2021 @10:48AM (#29393410)
That's at least partially why we have CV driven development be a thing in the industry. Either you jump on the NoSQL, AI, ML, blockchain, serverless, microservice, container, $FOO bandwagons, or you'll miss out on job opportunities and will have your CV stagnate.

That's not to say that those technologies don't have good use cases in particular circumstances, it's just that you as an IC probably don't have strong incentives to care about where the project will be in 5-10 years when you will have left for another company in 2-3. A lot of accidental complexity is an unavoidable reality when you have people who put their own careers ahead of most other concerns.

Re:Will Nix Overtake Docker? by oauea on Tuesday November 30, 2021 @08:36AM (#29391907)
Just build your own images like you would otherwise install the software on bare metal. Base those images on official images, not community images.
Re:Will Nix Overtake Docker? by kryptk on Tuesday November 30, 2021 @09:13AM (#29392267)
YMMV.

Personally, you would have to pry my Bitnami images out of my cold dead hands.. there is just no way my team of 2 can do anywhere near as good.

Re:Will Nix Overtake Docker? by oauea on Tuesday November 30, 2021 @12:39PM (#29394944)
Fair, there are a few high quality "vendors" that we also trust, and Bitnami is one of them. RandomJoe42/SomeImage is out, though.
Re:Will Nix Overtake Docker? by goodpoint on Tuesday November 30, 2021 @02:58PM (#29396722)
> Just build your own images

This is like saying "just wear a mask": it's not enough.

You rely on what other people are doing now and what they've done in the past.

This is an ecosystem problem, not an an individual problem.

Re:Will Nix Overtake Docker? by oauea on Tuesday November 30, 2021 @08:32AM (#29391875)
If you can do it in minutes on your machine, you can spend those minutes updating your Dockerfile to automate the steps instead. It's essentially the same thing.
Re:Will Nix Overtake Docker? by taeric on Tuesday November 30, 2021 @09:17AM (#29392313)
Unless auth is concerned. There are loads of tools I run that I want to just be me. Not whatever user is configured in the image.

And learning how to manage that mapping was a heck of a time sink.

Re:Will Nix Overtake Docker? by oauea on Tuesday November 30, 2021 @12:39PM (#29394934)
How you going to run those tools on CI? Or by your colleagues? These are all questions that need answering anyway, regardless of your usage of Docker.
Re:Will Nix Overtake Docker? by taeric on Tuesday November 30, 2021 @02:53PM (#29396638)
I already authed as me on my machine. Annoying to also have to auth as me in the container.

Or have we reached the stage where folks have secrets management for containers fully orchestrated at a personal user level?

Re:Will Nix Overtake Docker? by throwaway894345 on Monday November 29, 2021 @11:45PM (#29389061)
This mirrors my experience as a developer in an org that used Nix. If you want the dev team to have a strong dependency on the devops team for every little (often unpredictable) aspect of their workflow, Nix is the tool for the job.

Don’t get me wrong, I’m completely bought in on the vision of reproducible builds but there’s a long ways to go before it’s usable in real organizations. I’ve heard that some orgs manage it, but I really have no idea how. Maybe some critical mass of people who already know and love Nix (and implicitly packaging obscure dependencies)?

Re:Will Nix Overtake Docker? by hitpointdrew on Tuesday November 30, 2021 @08:15AM (#29391724)
> If you want the dev team to have a strong dependency on the devops team for every little (often unpredictable) aspect of their workflow,

I have never used nix, but from the article the author only concentrated on the fact that docker and nix create reproducible environments, and completely misses the other benefits of containers.

As a devops guy if someone hands me a nix project, how do I deploy that so it is highly available and scales by itself?

With containers I just put it in kubernetes.

Re:Will Nix Overtake Docker? by Nullabillity on Tuesday November 30, 2021 @08:35AM (#29391898)
You can use Nix as a better docker build, see https://grahamc.com/blog/nix-and-layered-docker-images or https://nixery.dev/.
Re:Will Nix Overtake Docker? by throwaway894345 on Tuesday November 30, 2021 @10:07AM (#29392887)
The dream is using something like Nix to not only reproducibly build a container image but also all of the infrastructure manifests which reference it. I _think_ this is achievable in Nix if you're willing to deal with all of the pain of actually using Nix; however, this would depend on pushing an image to a container registry as part of the Nix build and I'm pretty sure that violates Nix's idioms/conventions? I've certainly never heard of anyone building their entire infrastructure manifests this way.
Re:Will Nix Overtake Docker? by Nullabillity on Tuesday November 30, 2021 @10:23AM (#29393083)
> The dream is using something like Nix to not only reproducibly build a container image but also all of the infrastructure manifests which reference it. I _think_ this is achievable in Nix if you're willing to deal with all of the pain of actually using Nix;

We did that at an old job. Basically, Nix built images + K8s manifests + a script to push the images. Our CI job boiled down to `nix-build && ./result/push-images && kubectl apply -f ./result/manifests -R`. This is similar to how NixOS' nixos-rebuild boils down to `nix-build && ./result/activate`.

Re:Will Nix Overtake Docker? by verdverm on Tuesday November 30, 2021 @12:32PM (#29394805)
See https://dagger.io, the next gen tool by the creators of docker that does this.

CUE + BuildKit

Re:Will Nix Overtake Docker? by jokethrowaway on Tuesday November 30, 2021 @04:10PM (#29397498)
That's because nix is not a docker alternative.

I'd love a nix-like docker alternative to be fair but that'd be a massive undertaking.

Re:Will Nix Overtake Docker? by throwaway894345 on Tuesday November 30, 2021 @04:34PM (#29397772)
What is a "nix-like docker alternative"? Is that "using Nix to build container images"? Because Nix already does that.
Re:Will Nix Overtake Docker? by vineyardmike on Tuesday November 30, 2021 @01:25PM (#29395557)
Many orgs get reproducible builds by using different build tools and abstractions. Eg Amazon gets it via their Brazil tool.
Re:Will Nix Overtake Docker? by throwaway894345 on Tuesday November 30, 2021 @10:10PM (#29400226)
I was talking about Nix in particular, but yes, if you’re an enormous and highly lucrative tech company you can pay a team of engineers to build your own reproducible build tool. If the same team is very talented they can probably even get Bazel working. ;) But I don’t think any large orgs at all use Nix, presumably because it doesn’t scale beyond a small number of (enthusiastic) developers.
Re:Will Nix Overtake Docker? by lvass on Monday November 29, 2021 @10:23PM (#29388576)
I've always kept my usage of Nix language to a minimum and found it to be a joy every time. But it's no wonder you'd have trouble on something that humongous. Why did it get so large?
Re:Will Nix Overtake Docker? by exdsq on Monday November 29, 2021 @10:42PM (#29388695)
Nix can do everything and that's an issue - suddenly the CI servers, AWS deployments, build servers, testing, linting, dev package management, dev system environments, and more are all written in Nix packages. You need to write nix bindings. You need to add nix caching. And, as is fitting with a functional language, it can be beautifully (read: painfully) abstract. Some of the guys on the team were contributors to Nix so it wasn't like we weren't following best practices either.

I'm sure if everyone was willing to put in weeks/months of effort it'll be as fun to use as Haskell, but Nix isn't the thing I do so I need something simple - docker/docker-compose works perfectly well for that.

Re:Will Nix Overtake Docker? by lvass on Monday November 29, 2021 @11:09PM (#29388849)
Sounds like your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should. Would it be nicer if you just used Nix to build the application and a shell, in the simplest possible manner?
Re:Will Nix Overtake Docker? by exdsq on Monday November 29, 2021 @11:26PM (#29388950)
This is totally fine and I like it, my worry is scope creep once it’s in. Nix repl to experiment with new languages is really cool. Reversing updates to my OS is amazing. If I could have a rule that that is the limit - personal dev environments - but with additional support for docker then I’d be very happy.
Re:Will Nix Overtake Docker? by svenhy on Tuesday November 30, 2021 @07:27AM (#29391374)
No, Nix can't do everything, it can do the one thing it does, mainly it maintains immutable paths and allows to define derivations of any number of those paths using namespaced execution of build scripts. All this is controlled using expression in the Nix language, which is well suited for the intended purpose. It is in important aspect the opposite of Haskell. Nix does so well what it does, that it is easy to use it where better solution exists (looking at you NixOps)

So I agree with some parts of your criticism, on the other hand if Nix is be used to do the sensible subset of "CI servers, AWS deployments, build servers, testing, linting, dev package management, dev system environments, and more are", that would be great.

And what that subset is depends heavily on the given circumstances.

For me, being able to derive the "CI servers, AWS deployments, build servers, testing, linting, dev package management, dev system environments" things from a single source of truth, actually does sell Nix pretty nicely.

When approaching a complex, multi-component source repository, with the requirement for dev-environments, CI builds and generation of production ready Docker-images, a careful and not over-eager Nix build, seems a sensible choice, given that the layered nature of Dockerfiles and docker-compose obscures the DAG nature of built artifacts and dependencies. Nix also doesn't build anything by itself, it merely invokes low-level build tools in isolated environments that only contain the specified dependencies.

Sure, when using Nix one is forced to explicitly state the dependencies of each (sub-component-) build-step, that results in more code than a bunch of RUN ... calls in Dockerfiles. Both approaches have their sweetspots.

An investment into learning Nix is done once. And you are completely right, that refactoring to a Nix based build takes weeks in a serious legacy software project. I wonder myself, if this will be worth it. Currently, I think that might be the case, and that part of the weeks put into the build are not due to Nix and would have been necessary for Docker-based builds, too, and also one can do a lot of stuff much easier with Nix, which leads to "feature-creep" like "oh let me add this preconfigured vim/emacs/vs-code into the build env" that one would probably not do if it weren't so easy to customize everything with overlays and overrides.

But that is a good thing, and it is much better to have a capable tool and critical discussions with colleagues to limit the usage of that tool, than the other way around.

Heck, I remeber when we switched to a Maven build, and had a build with all bells and whistles. That took weeks, too, and all the generated artifacts, like source code documentation sites, were probably a waste if time to implement.

I am not sure if that proves or disproves powerful, declarative and modular build systems.

Dockerfiles and docker-compose.yamls have a learning curve too, but more importantly, if you have to invest more time per build maintenance than with a Nix based build, it is only a matter time until that costs more than the nix approach.

Re:Will Nix Overtake Docker? by aspaceman on Monday November 29, 2021 @10:44PM (#29388706)
I'm not the author and my opinion might not be warranted, but I would guess:

* When you modify your local nix file, there's only a couple things you need to change. It stays small. You can keep track of all these changes because they're in your head.

* The company needs a lot of little changes here and there so they build up. Foobar needs rooster v1,2 while Barfoom needs rooster v0,5. A lot different than managaging a users config. No longer possible to squeeze all the details into one brain.

This isn't to say Nix would be bad at handling any of these things. But I would agree with the OP. I have trouble teaching new devs how APT and shell work. Couldn't even imagine trying to explain Nix syntax to them.

The more Nix is used for, the more likely a dev will have to touch it. And teaching someone who doesn't care Nix is like teaching someone who doesn't care about Makefiles / Git. Doesn't work. They just get mad and say it's too complicated.

Re:Will Nix Overtake Docker? by rgoulter on Monday November 29, 2021 @10:50PM (#29388746)
> I've always kept my usage of Nix language to a minimum and found it to be a joy every time.

The trouble is, it's more significant to consider what the pain is in the worst case.

e.g. Two things which count against Nix: With stuff like Docker, I can get away with not really understanding what's going on, and have a quick + dirty fix. And with Nix, the community is smaller, so it's less likely you'll find an answer to the problem you're having right now.

I worked at a company where the DevOps team used Nix for the production/staging. For local development (and ephemeral development deployments), the dev team came up with a docker-compose setup. -- That said, there was at least one case where the devs contributed to the Nix stuff by way of copy-pasting to setup a new deployment.

Re:Will Nix Overtake Docker? by Barrin92 on Monday November 29, 2021 @11:05PM (#29388828)
>Build times in Haskell were bad enough

this one's very true. Every time I try to build something in Haskell on my laptop it feels like we're moving closer to the heat death of the universe. Is there some good read on how/why Haskell compilation times are so long compared to some other languages?

Re:Will Nix Overtake Docker? by Zababa on Tuesday November 30, 2021 @04:23AM (#29390402)
I can't offer an answer specific to Haskell, but I know that OCaml does some things to make the compilation fast: interface files, no circular dependencies, no forward declarations, not a lot of optimizations. From what I understand, those tradeoffs come from languages designed by Niklaus Wirth, where efficiency of compilation was important.

In general, I feel like every language that wasn't made to compile fast like Go, OCaml, Pascal and derivatives is going to be called "slow to compile". There's Java and C# that are kind of a middle-ground, since they emit bytecode for a JIT compiler. So my answer to "why Haskell compilation times are so long compared to some other languages?" would be "because compilation times don't take priority over some other points for Haskell and its users".

Re:Will Nix Overtake Docker? by funcDropShadow on Tuesday November 30, 2021 @10:19AM (#29393038)
Haskell compiles so slow because it needs way more optimizations then Ocaml before it performs good. The primary reason for that is its lazy evaluation semantics. Prior to GHC many researchers believed it to be impossible to execute lazy languages with comparable speed than imperative ones. Haskell and GHC were primarily developed as research tools. I.e. they valued the efficient exploration of new ideas in programming languages and implementations higher than the resulting language and implementation, at least in the beginning. And I would say Haskell was extremely successful considering those priorities.

That does not mean, that the implementation hasn't been moving towards industry-readiness for a long time.

Re:Will Nix Overtake Docker? by wyager on Tuesday November 30, 2021 @12:00AM (#29389128)
If you're compiling a project for the first time it's probably pulling in a ton of libraries and compiling them all from scratch. But let's be real here - we're talking maybe 3-5 minutes for the first build of a huge Haskell codebase, one or two orders of magnitude faster on second build.

It seems comparable to what I've noticed with Rust or C, and pretty fast compared to some medium/large C++ codebases I've built.

I'm not really sure what workflows people are using where this is a problem, so this may be less of a pain point for me than it is for you. Personally I build Haskell projects once to pull in the dependencies, then use ghcid to get ~instantaneous typechecking on changes I make to the codebase.

Re:Will Nix Overtake Docker? by AzzieElbab on Tuesday November 30, 2021 @12:22AM (#29389226)
From my experience with Haskell and Scala, derivations and not dependencies is what’s driving compile time. You can probably hide a crypto miner behind all that compiler magic. In Scala I usually move code that invokes a lot of derivations into sub projects, not a clue on how to do that in Haskell
Re:Will Nix Overtake Docker? by exdsq on Tuesday November 30, 2021 @05:48PM (#29398448)
I wish it took 3 - 5 minutes, I’ve built it projects that are far longer than that. I know this isn’t just me though, because the GHC build documentation tells you to go read stuff for a while as it builds. Longest I can remember is 2 hours.
Re:Will Nix Overtake Docker? by wereHamster on Tuesday November 30, 2021 @02:29AM (#29389821)
> moving closer to the heat death of the universe

So… you're saying your laptop is super cool? Because the heat death of the universe is when thermodynamic energy is equally distributed everywhere, which, given the large space of the universe, means really cold.

Re:Will Nix Overtake Docker? by xelxebar on Tuesday November 30, 2021 @04:39AM (#29390475)
Heh. Though while we're playing Pedantics, it's worth pointing out that the defining characteristic of Heat Death is maximum entropy, not uniform temperature. That said, it's entirely possible that the whole system of Energy Company + OP's computer actually does get cooler on average when compiling Haskell.

Here's a fun game. Have you ever played Follow the Energy? For example, depressing keys on your keyboard takes energy. Where does that energy come from? Well, the work performed by your fingers of course! But where does that energy come from? Well, the muscles in your fingers, of course! But where does that come from? Well, your food! But what about that? Well, your food's food! That? Eventually plants. That? The sun! That? Nuclear fusion. That? Gravitational potential! That? Etc...

But here's the funny thing. In this image, it kind of seems like energy gets "used up". The sun provides energy to the earth via solar radiation; the plants consume this energy; animals eat the plants, obtaining their energy; etc. However, energy is conserved. What gives?

Better yet, if the earth were not radiating just as much energy as it received, it should be heating up. However, the earth-atmosphere system is mostly constant temperature! This implies that the total energy flux is zero. If X Watts (Energy/Time) is coming in, then the earth actually radiates out X Watts as well.

So... if the total energy flux is zero, then how does your keyboard key actually get pressed? What gives?

The key is entropy. X Watts of solar radiation impinge upon the earth, but these photons are "hotter" (i.e. higher frequency) on average than those that radiate out. The balance is in numbers. You need more cooler photons to balance the energy of the hotter ones. E=hf; energy (E) is proportional (constant h) to frequency (f), after all.

This means that while energy is conserved, the flow of that energy increases the entropy of the system. In a very real sense, typing on your keyboard happens because of the waste heat generated in the process.

All driving us closer to Heat Death...

Re:Will Nix Overtake Docker? by MereInterest on Tuesday November 30, 2021 @07:30AM (#29391408)
> The sun! That? Nuclear fusion. That? Gravitational potential! That? Etc...

Minor pedantry on top of pedantry. The gravitational potential energy of a collapsing protostar gets you past the activation barrier for nuclear fusion, but isn't itself the source of energy beyond that.

Think of it like lighting campfire using friction. The friction heats up the kindling, but that investment allows you to access the potential chemical energy of the wood. The gravitational potential energy is converted into heat, and that investment allows you to access the potential nuclear energy of the unfused hydrogen.

Re:Will Nix Overtake Docker? by xelxebar on Wednesday December 01, 2021 @08:40PM (#29411502)
Thanks! That's one place in the chain that I paused for quite a while, deliberating what to write down. Overall, the little "causal chain" I wrote down is littered with little (but useful!) fibs, and the one you point out is probably the most egregious.

Nice analogy, btw.

Re:Will Nix Overtake Docker? by solohan on Tuesday November 30, 2021 @06:19AM (#29390978)
Thanks for elaborating on this. I guess the confusion also stems from the word "heat" which we usually associate with something that's warmer than some human day-to-day reference. Heat in the physical sense though just refers to the kinetic energy of a number of particles, which can be harvested (per the post above) provided there are differences in kinetic energy throughout space.

It's also interesting to play the "Follow the Energy" game to it's logical conclusion, namely that nearly all of the energy in the Universe (including that which you expend typing on your keyboard) originates from the gravitational potential created in the Big Bang (whatever that is, by the way). This begs the question; how was the entropy of the early Universe to low, that it could increase by such an enormous amount, to produce intelligent beings such as ourselves, typing things on a keyboard while we should be working?

It's really one of the most fundamental questions in cosmology, and one of the (many) reasons why I love physics.

Re:Will Nix Overtake Docker? by solohan on Tuesday November 30, 2021 @04:05AM (#29390299)
Actually, the heat death is when all potential energy in the universe has been converted to heat. So converting an excess of stored chemical energy in their laptop battery to heat by compiling a load of Haskell would be a fine way of increasing the entropy of the Universe. Thus moving ever so slightly closer to the inevitable heat death.
Re:Will Nix Overtake Docker? by nvarsj on Tuesday November 30, 2021 @08:30AM (#29391857)
I'd say don't blame the tool for what sounds like an architectural mess.

TFA points out using nix as a reproducible build environment - which it is excellent at. Create a shell.nix in your repo, and every dev uses the same exact tools within the comfort of their own machine/shell. Docker is much more painful for this kind of local dev workflow.

Re:Will Nix Overtake Docker? by exdsq on Tuesday November 30, 2021 @05:52PM (#29398477)
Potentially it was a mess, but the guys building it contribute to nix so I don’t think that’s to blame. Dev environments are okay, but that’s not the limit of nix - it gets used for build servers, ci servers, linting, replacing stack cabal yarn etc, creating docker, hell I’m pretty sure I saw nix creating nix
Re:Will Nix Overtake Docker? by kaladin-jasnah on Tuesday November 30, 2021 @06:30PM (#29398803)
I truly hope not as well. I spent an extensive period of time trying to learn Nix and integrate it into my homelab. Granted, it's not an enterprise environment, but it is the most painful, unstable, broken, buggy, bloated, hard-to-use, poorly-documented mess I've ever tried. I hope no one touches Nix with a ten-foot pole.

That's not to say it's not a great idea. It's just a huge pain to get to work, and the cost of trying to get a perfectly-working Nix environment is just *not* worth it. In my case, I was working with orchestrating virtual machines, which… well, there's a project called nixops that claims to do this. The thing is it flat-out didn't work half the time (also used python2.7 and I believe one of its dependencies were marked as insecure for a long period of time). I got so frustrated with this "advertised" nixops tool, that had to write my own replacement for nixops, and while it was a fun experience, I was so burnt-out from dealing with Nix and its breaking randomly every ten seconds, I just gave up on my homelab.

If you want any program that doesn't exist for Nix, you can't just use the program's build script. You have to manually make a Nix derivation, and then pray and hope it will actually compile. Want to deploy your Ruby on Rails application on NixOS? Prepare to spend three days trying to set this up, since there's only one other application and the entire process is poorly documented (even for a "regular" Ruby program that isn't Rails).

Without additional work, Nix will never be worth its cost (did I mention the interpreter took forever and sometimes failed on errors in files that had NOTHING to do with yours, leaving you to manually debug each line?). You could spend the days upon days upon days trying to get the damned thing to work for something far more useful instead. Since once you get Nix to work, it will break.

I'm sorry if I offended any Nix users/developers, but the product is just not ready for anything yet IMHO. I just don't have time to deal with it anymore, and I'd rather get on doing something more fun than dealing with a bloated, undocumented system that I can just replace with Docker and get my work done in 5 minutes instead of 5 days.

Re:Will Nix Overtake Docker? by pxc on Tuesday November 30, 2021 @06:48PM (#29398943)
> did I mention the interpreter took forever and sometimes failed on errors in files that had NOTHING to do with yours, leaving you to manually debug each line?

This is a common (and horrible) issue with dynamic languages that pass functions or blocks of code around. There have been some major improvements to Nix error messages which were included in the last release, and there's also ongoing work to address this through gradual typing.

There was a talk on the latter recently, with some examples that I think make the overall issue pretty clear: https://www.youtube.com/watch?v=B0J-SclpRYE

Re:Will Nix Overtake Docker? by kaladin-jasnah on Tuesday November 30, 2021 @09:17PM (#29399903)
> There have been some major improvements to Nix error messages which were included in the last release

That's good to hear (and I think I heard about it before). The problem is, there are still lingering issues with Nix, like how long it takes to figure out how to compile a new program, (exceptionally) poor documentation, packages being unmaintained, the nixpkgs repository being a gigantic blackhole that takes forever to eval, etc.

Don't get me wrong. I really like the idea behind Nix. Even with this fix though, I'm still not sure I would enjoy using Nix (since many, many other problems) or giving it another shot because of my emotional response to it that's been caused by burnout trying to wrangle with it.

Re:Will Nix Overtake Docker? by pxc on Wednesday December 01, 2021 @01:51AM (#29401373)
> I'm still not sure I would enjoy using Nix (since many, many other problems) or giving it another shot because of my emotional response to it that's been caused by burnout trying to wrangle with it.

I'm not sure you would, either, and I won't ask you to give it another shot right now. I understand the feeling. Maybe it's something to revisit after more time than has already passed.

Re:Will Nix Overtake Docker? by fulafel on Tuesday November 30, 2021 @02:38AM (#29389852)
Was/would it be hard to switch back to traditional containers or was there some kind of lock-in effect? Or was the consensus just pro-Nix?
Re:Will Nix Overtake Docker? by exdsq on Tuesday November 30, 2021 @08:01AM (#29391631)
The conesnsus with those who mattered to make that change was pro-Nix, but I have no idea how you'd go back to containers because by that point builds were declarative and the implementation was spread over abstractions and repositories! So if DevOps decided to quit because another company was using SuperNix2, which I think is a plausible thing if you have a team using a risky technology, then it would have required hiring consultants to do it - I think it would have been a months-long refactor going in blind.