Bare metal kubernetes is certainly a lot less complete out of the box when it comes to networking and storage but, people can, and often should, use a managed k8s service which provides all those things out of the box. And if you’re on bare metal once the infra team has abstracted away everything into LoadBalancers and StorageClasses it’s basically the same experience for end users of the cluster.

If you're talking about OpenShift on rented commodity compute, maybe. If you're talking about GKE/AKS/EKS or similar, I disagree wholeheartedly; you're then paying several multiples on the compute and a little extra for Kubernetes.

> because the one you picked in AWS doesn't support bare metal

That's just because AWS's Kubernetes offering is laughably bad.

There is huge difference in your experience whether you use Kubernetes via GKE (Autopilot) or any other solution (at least as long you don't have a dedicated infrastructure team).

I dunno, if you have to use GCP to have a good time, then that's a pretty great argument against k8s for a lot of people.

I think "bare" Kubernetes is still a quite nice tool that allows for learning transferable skills across clouds (similar to Terraform). E.g. even if I have to spin up my own nginx-ingress to be able to handle ingress resources, after having learned that initially, I can basically do the same thing across clouds.

It's just that GKE (Autopilot) does a lot of those out of the box for you that, so you get a much easier end-to-end experience for non-admins (= "request resources -> have them instantiated").