News for hackers. Stuff that matters.
Re:A note for LWN subscribers by jchw on Thursday January 20, 2022 @10:19AM (#30009929)
When running on Kubernetes or Lambda, you are still directly interfacing with the Linux kernel, unless you’re under something like gVisor. I also don’t think the endgame of “cloud” is that literally nobody knows how to run servers; in fact, Kubernetes is one of those technologies that makes bare metal much more appealing. I think cloud has outsized appeal for now simply because things are moving so quickly that it’s difficult to keep up, and I feel like that is probably not forever.

Even just for Kubernetes administration, it’s probably good to keep tabs on Linux, especially technologies like cgroups, even if your nodes are managed.

Re:A note for LWN subscribers by mschuster91 on Thursday January 20, 2022 @11:35AM (#30011151)
> in fact, Kubernetes is one of those technologies that makes bare metal much more appealing

Bare metal on Kubernetes is a PITA though to set up. For extremely small scale stuff in private labs, okay, that takes half an hour, but as soon as you are on the Internet, maybe have a couple rented bare metal machines, it gets nasty. For ingresses, you'll usually need to set up MetalLB which requires acquisition of a range of IP addresses and it only is capable of running one server (the "speaker") as the one and only entrypoint for all traffic. Not to mention you cannot have roles such as "internet-exposed node" and "not internet-exposed node", since MetalLB wants to run on all nodes.

Also, a Kubernetes environment will need at least four separate network ranges - a network for the Pod IPs, a network for cluster IPs of services, one for the nodes themselves and if you want to expose stuff to the Internet you need a fifth one that holds the public-routed IPs. This is really funny if you are in a corporate environment that uses all three of the historic private ranges (10.0.0.0/8,172.16.0.0/12,192.168.0.0/16), good luck carving out ranges that do not result in routing conflicts.

Mesosphere DC/OS was more usable in that regard as it had pre-defined the roles of nodes, included the load balancer as part of the platform and had only two IP ranges (node-internal and whatever the public IPs of the public nodes were)... whereas it's clear that the only designed/supported way to run a major Kubernetes deployment is to use a fully-managed Kubernetes provider. Everything else is a world of undocumented, spurious-bug-filled stuff and incomplete documentation.

Re:A note for LWN subscribers by champtar on Thursday January 20, 2022 @12:45PM (#30012110)
MetalLB is the best baremetal LB (for k8s) but far from perfect indeed. For the IP conflicts, there are other private IPv4 range that you can use, just pick something in the CGNAT range (100.64.0.0/10) or in 198.18.0.0/15 ("Used for benchmark testing of inter-network communications between two separate subnets")
Re:A note for LWN subscribers by mschuster91 on Thursday January 20, 2022 @12:49PM (#30012156)
The best thing about HN: You learn something new every day. Thanks!
Re:A note for LWN subscribers by dewey on Thursday January 20, 2022 @12:33PM (#30011945)
> Bare metal on Kubernetes is a PITA though to set up.

I think you are misunderstanding the previous comment. I read it as Kubernetes being so complicated (I enjoy it, but that's another discussion) to operate is making the regular bare metal operation more appealing. So people might choose the well understood bare metal over Kubernetes.

Re:A note for LWN subscribers by lima on Thursday January 20, 2022 @12:17PM (#30011739)
Even with gVisor you're still depending on things like the host kernel's scheduler and memory management.