This is the most frustrating but oft repeated nonsense about k8s.
Kubernetes is an amazingly helpful abstraction for any amount of load on any larger than trivially tiny set of services/storage/etc.
When you, like 90% of systems, don't have Google kind of load, you don't need Google number of nodes.
Machine count corresponds to load; abstraction doesn't.
Well, to be fair, yaml itself doesn't exactly have a stellar reputation, but I've never seen fifty lines of yaml that could convey so little information in any other place.
When you're writing YAML you can absolutely include from elsewhere (it calls it 'aliases', which is... Well, at least it exists) in multuple places - though annoyingly not from other files.
When you're reading (partially) generated YAML, (e.g. kubectl describe or edit) it's IMO a feature that everything is in its place with no indirection. (And as far as I'm aware this is YAML, not kubectl.)
K8s is developed to manage large number of service nodes running docker or oci containers coming from Google borg project. So it was designed for managing google level of service loads. It doesn't help when a service is only for few million users with few hundred or thousand containers. LXD and traditional tools can already do it without learning new way with lots of moving parts.
Well my company switched from VMs to k8s (few hundred containers) and it helped. Proof me wrong.
So in summary:
* we still have same amount of developers
* we still need same amount of sysadmins
* we introduced more complexity
* the application became slower
* the whole thing became much more expensive (managed k8s price, education, consulting, etc.)
* but! we have the same amount of income every month. Surprisingly our users don't pay more for the same service because we run our stuff on k8s
Abstractions have a cost. You either pay the cloud provider or you pay the Ops team that runs the cluster.
If you're fine with that and have evaluated the costs correctly, great!
In my current work, we run several Kubernetes clusters via GKE on Google Cloud Platform. We're a tiny company — less than 20 nodes running web apps, microservices and search engines — but we're benefiting hugely from the operational simplicity of Kubernetes. Much, much, much better than the old fleet of Puppet-managed VMs we used to run. Having surveyed the competition (Docker Swarm, Mesos/Marathon, Rancher, Nomad, LXD, etc.), I'm also confident that Kubernetes was the right choice.
Kubernetes may be a large and complex project, but the problem it solves is also complex. Its higher-level cluster primitives are vastly better adapted to modern operations than the "simple" Unix model of daemons and SSH and what not. The attraction isn't just the encapsulation that comes with containers, but the platform that virtualizes physical nodes and allows containers to be treated as ephemeral workloads, along with supporting primitives like persistent volumes, services, ingresses and secrets, and declarative rules like horizontal autoscalers and disruption budgets. Given this platform, you have a "serverless"-like magically scaling machine full of tools at your fingertips. You don't need a huge workload to benefit from that.
As for LXD and the other alternatives, these might be great, but Kubernetes has vastly more mainstream adoption these days. That's more important than mere technological superiority, or everyone would be using OpenSolaris with Zones and ZFS these days.
Kubernetes is an Overkill for majority of development and production systems.
So the win is not having to manage something :-) How is that a fair comparison?
Exactly!
> How is that a fair comparison?
Not only is it a fair comparison, but I'd say it's the entire raison d'etre of using any orchestration platform: You get to farm out bits of the infrastructure to others. Getting on k8s enables you to not have to run your own infrastructure, whereas having VMs being your abstraction sorta by definition does not.
Hell, the stock market goes offline every night.
Those last % of uptime are very, very expensive at every level (money, people, logistic...). They better be worth it.
HA is not just about getting that sixth 9, it's also about the peace of mind that you are insulated from a range of problems. Not only would I not worry about our SQL service going down, I also don't have to worry about what I have to do when a node goes down, because that's automated.
But the cost of a 500 is always to be balanced with the cost of going full throttle with micro services.
As perfectionists, it's easy to fall into the trap of choosing the later by default.
2. Don't spend too much on it just use traditional knowledge and, you can still use bare-metal, VM or LXD container. Focus on application not on programming a tool designed to solve problems for Google size deployment.
3. Kubernetes is a completely different way of doing things. So if you want to leverage old cluster and distributed system knowledge you can use VM or LXD containers. Kubernetes is an overkill.
4. Abandon kubernetes instead of complaining, just use simple systems.