Kubernetes reinvented virtual machines in a good sense

Re:Kubernetes reinvented virtual machines in a good sense by moondev on Sunday July 31, 2022 @08:55PM (#32300547)

Even if Windows containers are not as portable, the ability to manage them from the same Kubernetes controlplane that manages Linux containers seems pretty powerful. It also enables you to do it outside of Azure/Service Fabric on other clouds or metal.

https://kubernetes.io/docs/concepts/windows/

https://docs.microsoft.com/en-us/azure/aks/windows-faq?tabs=...

[ Parent | on HN ]

Re:Kubernetes reinvented virtual machines in a good sense by jiggawatts on Sunday July 31, 2022 @09:01PM (#32300594)

You're trading one convenience for a world of hurt if you use Kubernetes with Windows. Just take a casual glance at the "Windows containers" issues list on GitHub: https://github.com/microsoft/Windows-Containers/issues

Some of the very recently closed issues were jaw-dropping, such as totally broken networking in common scenarios.

DNS resolution is very different in Windows compared to Linux, making much of the "neatness" of Kubernetes pod-to-pod communication not work.

There is no maximum memory limit in Windows kernel "job objects" (equivalent to cgroups), so one memory leak in one pod can kill an entire node or cluster. This is very hard to solve, and I've seen it take out Service Fabric clusters also.

Etc, etc...

[ Parent | on HN ]

Re:Kubernetes reinvented virtual machines in a good sense by moondev on Sunday July 31, 2022 @09:20PM (#32300728)

Thanks for sharing, it all seems terrifying but still very interesting to explore. I was curious how the Windows nodes would even join - turns out they support cloud-init just like Linux machines.

https://cloudbase.it/cloudbase-init/

In theory this means they could be managed then by something like KubeVirt instead of treating them like a worker node. https://kubevirt.io/user-guide/virtual_machines/startup_scri...

Excited to see this space continue to evolve for sure.

[ Parent | on HN ]

Re:Kubernetes reinvented virtual machines in a good sense by jiggawatts on Sunday July 31, 2022 @11:08PM (#32301507)

Cloudbase-init is interesting, but Microsoft doesn't support or (apparently) use it. There's no mention of it in their docs: https://www.google.com/search?q=site%3Adocs.microsoft.com+%2...

New Windows VMs are generally built using PowerShell DSC or a similar native tool.

[ Parent | on HN ]