Spinning up a K8s cluster "in the cloud" is easy and everyone jumps on that happy "look how simple it all is" bandwagon, forgetting than it's just the beginning of a very long journey. There are millions of blog articles of varying quality that explain how easy it is, because it's very simple to spam search engines retelling a story how to click a couple buttons or do some basic Terraform/CloudFormation/whatever.
And here's what they don't tell you - maintaining this machinery is still your business, because all you get is a bunch of provisioned node machines and a cookiecutter to spin it up. Plus a bunch of scripts to handle most common scenarios. The rest is either yours or tech supports' (if you pay extra for it).
If you're at any decent scale (looks like you are), then switch to GKE, or switch to on-prem and buy some hardware + a Kubernetes distro like Mirantis/Openshift/Tanzu.
Heck, go run k3s on Hetzner and you won't have that much more work, but save literally millions at the scale you're talking about.
The moment Hetzner starts providing managed DBs or managed kubernetes, they will wipe the floor with all the big 3...
Now this is true for doing it without k8s too, but somehow there was never a huge set of blog posts about "it's really hard to set up a load balancer and a secrets service and networking" but there is for k8s, so there must something intrinsic in either its design or its documentation that is causing that. I think it's probably that k8s is designed for google-scale deployments, so for most people the initial burst of complexity is a bit overwhelming.
We spin up EKS. We install the newrelic and datadog log ingestion pods onto it, provided in a nice "helm" format.
We install a few other resources via helm, like external secrets, and external dns, and a few others.
Kubernetes EKS runs like a champ. My company saves 100k/mo by dynamically scaling our cloud services, all of which are running on Kubernetes, to more efficiently use compute classes.
My company has over 50 million unique users monthly. We have massive scale. Kubernetes just works for us and we only have 2 people maintaining it.
What we gain is a unified platform with a consistent API for developing our services. And if we wanted to migrate elsewhere, it is one less thing to worry about.
¯\_(ツ)_/¯
Feels like some kind of hipster instinct to dislike the "cool new thing"... even though k8 has been around for years now and has been battle tested to the bone.