News for hackers. Stuff that matters.
Re:Warewulf is a stateless and diskless container OS provisi... by PAPPPmAc on Thursday March 06, 2025 @03:55PM (#43284987)
I've been using Warewulf (&co.) for provisioning bare-metal clusters for decades (back into the Perceus days between Warewulf 1 and 2), it's a solid easy-to-comprehend tool that does things in ways that are transparent and built from generic [u/li]nux tools enough that they're not hard to think about when needed, but automated enough you usually don't have to.

Definitely shows its research roots, best-tested with RHEL-alikes, reasonably well tested with Suse and Debian, and you may be in for some extra work if you need provision something else, but that pretty much covers the common cases (and it integrates with containerization tools if you need some specific environment on the nodes).

It's a nice to have when you need to spin many nodes.

Re:Warewulf is a stateless and diskless container OS provisi... by generalizations on Thursday March 06, 2025 @05:00PM (#43285488)
Why use warewulf? Seems like there have got to be simpler solutions for dealing with bare metal clusters than all of this.
Re:Warewulf is a stateless and diskless container OS provisi... by anderbubble on Thursday March 06, 2025 @05:04PM (#43285518)
Perhaps you have not seen some of the other solutions out there in this space.

Warewulf _is_ the simpler solution.

Re:Warewulf is a stateless and diskless container OS provisi... by mikestorrent on Friday March 07, 2025 @11:46AM (#43291702)
Always feels like it will be simpler... you start with some iPXE, start building, and 6 months later you have a poor imitation of a product like this that works only for your specific use cases and causes you a headache if the company pivots and you have to make it do something new.

Been there, built that. Next time I'm using something with a community, and if it doesn't do what I need, I'm contributing upstream until it does.

Re:Warewulf is a stateless and diskless container OS provisi... by generalizations on Friday March 07, 2025 @12:51PM (#43292331)
Ooof sounds like experience talking. I guess flexibility is kinda paramount here. Do you think you'd use something like warewulf then?
Re:Warewulf is a stateless and diskless container OS provisi... by superb_dev on Thursday March 06, 2025 @05:48PM (#43285816)
It’s that old? I can’t believe it took me this long to find Warewulf! I’ve tried the more complex solutions and this looks like what I’ve always dreamed of
Re:Warewulf is a stateless and diskless container OS provisi... by MortyWaves on Thursday March 06, 2025 @03:51PM (#43284953)
Where does this fit in the Ansible + PXE boot vs Terraform vs NixOS scale? Seems to be within that space, but before the "infrastructure as code" phrase was coined.
Re:Warewulf is a stateless and diskless container OS provisi... by anderbubble on Thursday March 06, 2025 @05:01PM (#43285493)
It's PXE boot for mostly stateless node / disk images, with a template-based overlay system for customizing the image as it's applied to the node.
Re:Warewulf is a stateless and diskless container OS provisi... by generalizations on Thursday March 06, 2025 @07:29PM (#43286455)
So with the overlay you can make node-specific changes? I was reading through the initial setup guide but I couldn't figure out how you actually specify which node gets which overlays (or one-off edits of whatever kind).
Re:Warewulf is a stateless and diskless container OS provisi... by mooreds on Thursday March 06, 2025 @02:21PM (#43284102)
Re:Warewulf is a stateless and diskless container OS provisi... by bluedino on Thursday March 06, 2025 @04:26PM (#43285224)
And if you want to pay for support you can reach out to CIQ

https://ciq.com/products/warewulf/

Re:Warewulf is a stateless and diskless container OS provisi... by mkesper on Thursday March 06, 2025 @03:19PM (#43284690)
Weird this does not make use of IPv6. I'd thought this was a given if you have tens of thousands of nodes.
Re:Warewulf is a stateless and diskless container OS provisi... by 8organicbits on Thursday March 06, 2025 @03:26PM (#43284761)
10.0.0.0/16 supports 64k hosts, so it seems it would fit in IPv4.
Re:Warewulf is a stateless and diskless container OS provisi... by anderbubble on Thursday March 06, 2025 @05:11PM (#43285569)
That's the thing: people don't put their HPC clusters on the public Internet; so an internal IPv4 network just keeps being fine.
Re:Warewulf is a stateless and diskless container OS provisi... by ale42 on Friday March 07, 2025 @06:06AM (#43289209)
True, but on the other hand it might be that all private use blocks are already in use (10.0.0.0/8 is totally in use in our internal LAN), so if I want the nodes to reach those private IPs, I can't assign the same block. And we do have services on IPv6.
Re:Warewulf is a stateless and diskless container OS provisi... by wmf on Thursday March 06, 2025 @04:25PM (#43285210)
It's from 2001 when IPv6 didn't really exist. I love IPv6 but it's basically crackpot retrocomputing tech at this point.
Re:Warewulf is a stateless and diskless container OS provisi... by generalizations on Thursday March 06, 2025 @04:33PM (#43285284)
Can't decide if the retrocomputing you mean is the warewulf or the ipv6.
Re:Warewulf is a stateless and diskless container OS provisi... by wmf on Thursday March 06, 2025 @04:44PM (#43285367)
IPv6 is retrocomputing. I don't know much about Warewulf.
Re:Warewulf is a stateless and diskless container OS provisi... by generalizations on Thursday March 06, 2025 @04:59PM (#43285471)
Huh. Did not realize we'd given up on that. Did the industry just settle on IPv4 after all?
Re:Warewulf is a stateless and diskless container OS provisi... by mcpherrinm on Thursday March 06, 2025 @05:07PM (#43285539)
Really depends on who you ask. You still need v4 to be "globally reachable", but v6 is optional.

AWS seems to finally be feeling the pinch of IPv4 exhaustion and is pushing v6 support everywhere now, and starting to charge for v4.

Mobile networks already have, and many are natively IPv6, with NAT64/464XLAT or other tech for bridging to v4. Apple's App store requires apps to support IPv6-only networks.

CDNs and clouds etc mean that websites don't even really need to worry about their own IP allocation, and just let their provider figure out exposing things worldwide.

Re:Warewulf is a stateless and diskless container OS provisi... by joshstrange on Thursday March 06, 2025 @06:13PM (#43285991)
> Apple's App store requires apps to support IPv6-only networks.

I read that and thought "huh, is that recent?" and found posts that were 9 years old about it. I guess apps just have to work on an IPv6-only network but I'm honestly surprised my apps do. I don't test in IPv6, my home network has it disabled, most of my servers don't have anything for IPv6 that I know of. Odd.

Re:Warewulf is a stateless and diskless container OS provisi... by ianburrell on Thursday March 06, 2025 @06:19PM (#43286024)
If you use the official network API, don't go out of way to use the old one, any app should support IPv6.
Re:Warewulf is a stateless and diskless container OS provisi... by Arnavion on Friday March 07, 2025 @10:49AM (#43291125)
As long as the the servers they talk to have v6 addresses. Or does Apple implement some DNS64+NAT64 / 464XLAT setup transparently in its OSes too?
Re:Warewulf is a stateless and diskless container OS provisi... by mcpherrinm on Friday March 07, 2025 @12:19PM (#43292005)
I think Apple tests on "IPv6 Only" networks that have DNS64/NAT64 to reach v4 servers
Re:Warewulf is a stateless and diskless container OS provisi... by jcgl on Friday March 07, 2025 @03:55PM (#43294536)
They do implement a CLAT, both in iOS and macOS afaiu.
Re:Warewulf is a stateless and diskless container OS provisi... by aitchnyu on Friday March 07, 2025 @08:27AM (#43289963)
Tangential, will email host spam reputation rely on IPv4 scarcity forever?
Re:Warewulf is a stateless and diskless container OS provisi... by dspillett on Friday March 07, 2025 @08:52AM (#43290149)
For longer, I expect. For a long time email has been partially centralised so for most real people and a lot of systems mail goes out through a specific host (or small number of hosts) on the edge of their network or completely outside it (sending individuals sending via services like gmail, and systems using services like sendgrid, and so forth) so the need to push for IPv6 is less apparent for mail sending than a number of other things. There are orders of magnitude less hosts sending mail than, say, making HTTP(S) requests.
Re:Warewulf is a stateless and diskless container OS provisi... by anderbubble on Thursday March 06, 2025 @05:02PM (#43285499)
We're always looking towards IPv6 support! And you _can_ do it today, with a little bit of work. But it's been difficult to prioritize in the main project when so few of our users (read: maybe one) have expressed interest.
Re:Warewulf is a stateless and diskless container OS provisi... by withinboredom on Friday March 07, 2025 @05:14AM (#43288912)
> when so few of our users

existing users, I suppose. New users are looking for exactly this feature and will walk away. So, you can now count it up to 3.

IPv4 has gotten quite expensive. A newer company I'm working with doesn't even have IPv4 access past the edge. There is just a little proxy that handles IPv4 translation on the edge; it barely gets any traffic.

Re:Warewulf is a stateless and diskless container OS provisi... by sneak on Thursday March 06, 2025 @06:42PM (#43286157)
This project’s README tells me so little.

https://sneak.berlin/20241224/readme-howto/

Re:Warewulf is a stateless and diskless container OS provisi... by xenophonf on Thursday March 06, 2025 @09:27PM (#43286960)
The documentation link is _right there_.
Re:Warewulf is a stateless and diskless container OS provisi... by sevg on Friday March 07, 2025 @02:02AM (#43287863)
The words in blue that are underlined are called hyperlinks. You can click them ;)

This readme howto is truly way too long and excessively prescriptive, and the author goes too far with his inserting his opinions (ie, don’t use github, don’t use discord etc.). I couldn’t possibly recommend this howto.

Re:Warewulf is a stateless and diskless container OS provisi... by mrbluecoat on Thursday March 06, 2025 @08:34PM (#43286751)
Is it just for short-term HPC compute job use cases or also IoT bare metal with OTA upgrades?
Re:Warewulf is a stateless and diskless container OS provisi... by CaliforniaKarl on Thursday March 06, 2025 @09:45PM (#43287037)
Its main thing is taking a system that’s just powered on, and giving it an operating system. Said system can then run HPC jobs (or shell sessions, or web sites, or data-transfer); until it’s rebooted, and the cycle begins again.

I don’t know how well it would work in IoT: The device needs to PXE-boot, which requires support from the DHCP server and the hardware boot environment (UEFI).

Re:Warewulf is a stateless and diskless container OS provisi... by mikeytown2 on Thursday March 06, 2025 @09:43PM (#43287029)
Re:Warewulf is a stateless and diskless container OS provisi... by 4ggr0 on Friday March 07, 2025 @01:46AM (#43287823)
Right now I'm searching for the optimal solution to host containers in a small but cool way. Thinking about just using a plain Linux host, configured via ansible and compose.

Used Portainer so far, but that's a bit bloated for my simple use (one host, no-HA, lab env). Kubernetes is way too complex as well.

Warewulf sounds fun to try :D all of my profiles would probably only have one node. Does Warewulf make a fraction of sense when having a tiny, quasi-local environment?

EDIT: ah, nevermind. stateless and temporary makes no sense for my usecase, as my containers will run 24/7 with rare changes. But I will think about Warewulf if I ever dive into large-scale containerization :)

Re:Warewulf is a stateless and diskless container OS provisi... by huijzer on Friday March 07, 2025 @02:09AM (#43287884)
If you want small and cool, Wasmtime should be mentioned. It’s a WebAssembly runtime to it can only run WebAssembly programs, which narrows the set of programs that can be executed. However, if you can get your program in WebAssembly then Wasmtime is very cool. Startup time is a few ms versus hundreds of ms for Docker. Memory usage is also orders of magnitude less. And security guarantees are similar.
Re:Warewulf is a stateless and diskless container OS provisi... by omnimus on Friday March 07, 2025 @02:41AM (#43288009)
Maybe Dockge? Its like Portainer light/cool.
Re:Warewulf is a stateless and diskless container OS provisi... by 4ggr0 on Friday March 07, 2025 @02:30PM (#43293516)
Also great suggestion, this looks like a nice addition to the stack! thx :)
Re:Warewulf is a stateless and diskless container OS provisi... by proxysna on Friday March 07, 2025 @06:56AM (#43289451)
> plain Linux host, configured via ansible and compose.

There is your answer. I'd would also add watchtower to keep your containers up to date.

Re:Warewulf is a stateless and diskless container OS provisi... by 4ggr0 on Friday March 07, 2025 @11:31AM (#43291562)
great suggestion! thanks, will definitely implement this :)
Re:Warewulf is a stateless and diskless container OS provisi... by brazzy on Friday March 07, 2025 @07:44AM (#43289714)
> Thinking about just using a plain Linux host, configured via ansible and compose.

That's what I've been using since 2019 (plus caddy as reverse proxy for various web services), and in general I've been happy. Upgrading to docker-conpose-v2 caused me some headaches recently though, and I'll soon have to upgrade the underlying Ubuntu server, which I am dreading.

Re:Warewulf is a stateless and diskless container OS provisi... by udev4096 on Friday March 07, 2025 @07:53AM (#43289766)
If you really want a lightweight experience, go with alpine and then run all the daemon processes using supervisor [1] and for HA, you can use keepalived which uses VRRP for HA

[1] - https://github.com/Supervisor/supervisor

Re:Warewulf is a stateless and diskless container OS provisi... by roygbiv2 on Friday March 07, 2025 @07:11PM (#43296213)
Nomad is very cool for hosting containers. Not nearly as complex as kunernetes and you can scale to multiple hosts if needed easily.
Re:Warewulf is a stateless and diskless container OS provisi... by suzzer99 on Friday March 07, 2025 @02:47AM (#43288032)
Did anyone else read this as "stateless and dickless" at first and wonder what the heck was going on?

No?

I'll show myself out.

Re:Warewulf is a stateless and diskless container OS provisi... by yreg on Friday March 07, 2025 @05:07AM (#43288866)
I did as well, "iskl" seems like an unsual english char sequence, at least compared to "ickl".
Re:Warewulf is a stateless and diskless container OS provisi... by naikrovek on Friday March 07, 2025 @11:19AM (#43291434)
> Warewulf is a stateless and diskless container OS provisioning system

are we just always assuming that everything is Linux, now? There are other operating systems, and this one only supports Linux.

I know that once you start working in Linux a lot, you start to ignore other operating systems, but they still exist.

This is not an "OS provisioning system", this is a "Linux provisioning system".

Re:Warewulf is a stateless and diskless container OS provisi... by linuxftw on Friday March 07, 2025 @11:53AM (#43291771)
If we're going to be pedantic, Why not GNU/Linux, as Linux is just the kernel? Of course, not all modern distros are based on the GNU userlands.
Re:Warewulf is a stateless and diskless container OS provisi... by yjftsjthsd-h on Friday March 07, 2025 @01:07PM (#43292487)
Linux is a kernel, not an operating system; it's a system for provisioning Linux-based OSs.
Re:Warewulf is a stateless and diskless container OS provisi... by yanko on Saturday March 08, 2025 @01:31AM (#43298005)
https://www.rocksclusters.org/#

One more good project from the very past