Re:Kubernetes reinvented virtual machines in a good sense
by moondev on Sunday July 31, 2022 @09:20PM
(#32300728)
Thanks for sharing, it all seems terrifying but still very interesting to explore. I was curious how the Windows nodes would even join - turns out they support cloud-init just like Linux machines.
https://cloudbase.it/cloudbase-init/
In theory this means they could be managed then by something like KubeVirt instead of treating them like a worker node. https://kubevirt.io/user-guide/virtual_machines/startup_scri...
Excited to see this space continue to evolve for sure.
Re:Kubernetes reinvented virtual machines in a good sense
by jiggawatts on Sunday July 31, 2022 @11:08PM
(#32301507)
Cloudbase-init is interesting, but Microsoft doesn't support or (apparently) use it. There's no mention of it in their docs: https://www.google.com/search?q=site%3Adocs.microsoft.com+%2...
New Windows VMs are generally built using PowerShell DSC or a similar native tool.
Some of the very recently closed issues were jaw-dropping, such as totally broken networking in common scenarios.
DNS resolution is very different in Windows compared to Linux, making much of the "neatness" of Kubernetes pod-to-pod communication not work.
There is no maximum memory limit in Windows kernel "job objects" (equivalent to cgroups), so one memory leak in one pod can kill an entire node or cluster. This is very hard to solve, and I've seen it take out Service Fabric clusters also.
Etc, etc...