k8s is not the right tool for every job. Most companies are not at the scale where they need to worry about the problems that it's trying to solve. But it's a cargo cult - they see the blog posts about how moving to k8s solves a bunch of problems that come up as you scale and decide they need to be solving all of those problems also even though there are simpler solutions at their current scale.
There's a bunch of other platforms out there that are way more opinionated and less "awkward" but they don't have the buzz that k8s has.
Kubernetes ought at least to be the right tool for a wider range of compute task. And I think it could have been the right tool for a wider range of compute tasks.
For instance, I don't regard one of our products that runs in three availability zones and has 3-4 instances per AZ as being "large scale". It is still a small system. And it doesn't run in multiple AZs for performance reasons but because we really need high availability.
We embedded discovery, failover and automatic cluster management in the server software itself. But it isn't really how we'd like to do it. But it is still less of a hassle than running K8S. (It also means that we can do that if you license our software to run it on-prem on pretty much most runtime environments, and that has its value, but again, this isn't functionality you want or should have to do yourself)
At ~12 instances probably still in "pets aren't so bad" territory.
Of course, there's a huge cost to building your own framework as well... And it's easy to get wrong.
I started https://www.jetpack.io/ recently to try and build a better solution on top of Kubernetes. If you're interested in checking it out and giving us early feedback, you can signup on the website, or email us at `demo at jetpack.io`
Pretty much every piece of software I've written in the past decade that tends to have configs in production can also work with little or no config. Including clustered software that just exploits things that are widely available like Zeroconf to get a decent low effort cluster up and running. No, you probably won't be able to (or want to) use those features in production, but that's beside the point. The point is to lower the thresholds. And then keep aiming at lowering them wherever you can because asking other people to care about your software is seriously uncool. Other people will never care as much as you do.
It is normal for programmers to become defensive about software. Be it their own or software they really like. But it is far more productive to assume that when users think something is awkward, perhaps it is because it is awkward. And perhaps it could have been done better.
Nobody actually gives a crap what someone thinks Kubernetes was built for -- and what kind of rubbish experience they think is deeply justified by the goals.
It is either needlessly awkward to use or it isn't. And guess what: most people think it is awkward. And I seriously doubt it needs to be this awkward.