There's two things there - the complexity of engineering the cluster itself, and the complexity of using it.

The former is where the pain is. If you can remove that pain by using a reliable managed offering, it changes the perspective a bit.

The complexity of using it is also non-trivial - you have to learn it's terminology and primitives, understand the deployment and scheduling model, know how volumes are provisioned, aggregate logs, etc.

However, the ROI for learning that complexity can be pretty good. If you get comfortable with it (maybe a month or so of learning and hacking?) you get sane rolling deployment processes by default, process management and auto healing, secrets and config management, scheduled tasks, health checks, and much, much easier infrastructure as code. Which means if things do go really sideways, it's usually not that hard just to stand up a replacement cluster from scratch. With a bit more reading and some additional open source services, you get fully automatic SSL management with let's encrypt too.

All that said, I absolutely agree with you in principle. No one should overcomplicate their deployments for no benefit. It's just worth reflecting on what those benefits are. Kubernetes had a bunch of them - and whether they're worth the effort of getting to know it depends very much on the system(s) you're running.

Thanks for an insightful response - Don't you think that a lot of wizbang software developers just want to use latest and greatest buzzword thing, whatever that might be, to get some cool points? As I grow older, I see increasing lack of objectivity and decreasing attention to KISS principles.

Yes, absolutely, and it drives me up the wall. I've seen some incredibly unsuitable technology choices made, the complexity of which have absolutely sunk productivity on projects.

That said, it's easy to become cynical and associate anything that's become recently popular with hype-driven garbage. But that can blind you to some really great new stuff too. I tend to hang behind the very early adopters and wait to see how useful new tech becomes in the wild - the "a year with X" style blog posts tend to be really informative.

The point I'm trying to make is that people often think in absolute terms. The problem isn't thinking "React is good for SPAs" - it's thinking "React is good for all websites". I've come across a number of engineers now who genuinely think React is a good fit for e.g. totally static sites, "because it's fast".

I've come across similar persistent beliefs around NoSQL databases. Some teams wouldn't begin to consider an RDBMS for their primary data store, even while their data is heavily relational - because "SQL databases don't scale".

It's not that technologies are good or bad - it's the blind belief that a given technology is best for all situations. The hype drives the development. Not of the underlying tech itself, but in the teams using that tech.

If the same company had to hire someone else with your same skillset as they expand or replace a coworker, they would pay them market value. They would have to offer competitive salary. When that happens, you find new employees making more than equally skilled existing employees. It’s called salary inversion.

but, the company I worked for was not paying higher wages for the newer employees than myself and I got pretty regular, pleasant raises.

I bet they talk about the company “being like a family”, don’t they?

It remains the best company I’ve ever worked for. Small and my opinion and actions actually had real impact to core features of the company, so :)

Worth it, I think.

I did have direct influence into huge and important parts of that business, though and I really, actually was respected and given broad rein in what I could do. My coworkers and I also tended to have a great time working, and I had direct lines to the CTO. We talked often.

  If you're not going to build the next Facebook, why would you need so much complexity?

I don't find them complex at all. You just tell the tools to be in a specific state and the tool applies the necessary changes. Server templates. Provisioning. Orchestration. etc.

That being said, I always recommend using a tool like Terraform to back up infrastructure and the likes.

The point I wanted to make is that my opinion is a bit different. Being able to declare how state should be instead of doing it imperatively/with configuration management is just something I enjoy and which I think does not cost much more in comparison.

That is why I wondered why not use it as a small startup?

Said applications don't have to be applications that you're writing as part of your startup. It can be ElasticSearch cluster, Redis, tooling to run some kind of ETL that happens to be able to reuse your k8s cluster for task execution, CI/CD, etc. etc.

Once you don't personally have your hand in every application your company is running (in addition to the points the other comments have brought up).

When you buy a large instance, you still need to set up the instance and tweak it to your application's needs. You then need to babysit this node.

It's called "use kickstart"

They are mostly plain old PHP webapps, non-trivial amount of them Wordpress (shudduer), some done in random frameworks, some with ancient dependencies, some in node.js, one was ruby, etc. They are the equivalent of good old shared hosting stuff.

With kubernetes, we generally manage them in much simplified form, and definitely much cheaper than hosting multiple VMs to work around issues in conflicting dependencies at distro level or keeping track of our own builds of PHP.

We also run CI/CD on the same cluster. Ingress mechanics mean we have vastly simplified routing. Yes, we cheat a lot by using GKE instead of our own deployment of k8s, but we can manage that too, it's just cheaper that way.

Pretty much smooth sailing with little operational worries except the fact that Google's stackdriver IMO sucks :)

I wasn't saying that just use 24 core single instance. Perhaps I should have worded it better.

You can use GCP control panel to launch clusters, add nodes, launch containers expose and autoscale them without touching a single line of config file or a terminal if thats what you like. If you launch / manage your own cluster though.. It’s a pain.

How do you coordinate rolling deployments?

(I'm not saying you need Kubernetes to do these things. But if you've written something to handle them, it is very probably Kubernetes-shaped).

Helps if you also don't do due diligence on various things (like actually caring about logging and monitoring), so you scratch out those concerns. This is not even half sarcastic, people went pretty far with that.

You might probably still spend more time than you think on actual deployment and management due to lack of automation & standardization, but if you're sufficiently small on the backend it works.

When the number of concerns you have to manage rises and you want to optimize the infrastructure costs, things get weirder. Kubernetes' main offering is NOT scaling. It's providing a standardized control plane you can use to reduce the operational burden of managing often wildly different things that are components in your total infrastructure build - Things like infrastructural metrics and logging, your business metrics and logging, various extra software you might run to take care of stuff, making it easier to do HA, abstract away the underlying servers so you don't have to remember which server hosted which files or how the volumes were mounted (it can be as easy as classic old NFS/CIFS server with a rock-solid disk array, you just don't have to care about mounting the volumes on individual servers). It makes it easier to manage HTTPS routing in my experience (plop an nginx-ingress-controller with a host port, do a basic routing setup to push external HTTP/HTTPS traffic to those nodes that run it, get the bliss to forget about configuring individual Nginx configs anymore --- or use your cloud's Ingress controller, with little configuration difference!).

In my experience, k8s was a force multiplier to any kind of infrastructure work, because what used to be much more complex, even with automation tools like Puppet or Chef, now had a way to be nicely forced into packages that even self-arranged themselves on the servers, without me having to care about which server and where. Except for setting up a node on-prem, or very rare debug tasks (that usually can be later rebuilt using k8s apis to get another force-multiplier), I don't have to SSH to servers except maybe for fun. Sometimes I login to specific containers, but that falls under debugging and tweaking the applications I run, not the underlying infrastructure.

That's the offering - whether it is right for you, is another matter. Especially if you're in cash-rich SV startup, things like Heroku might be better. For me, the costs of running on PaaS are higher than getting another full-time engineer... and we manage to run k8s part-time.

Additionally, new crazy things are emerging like service mesh (Istio, Linkerd) which make observability and security easier.

Of course, it all is getting quite complex. But it brings value in the end.

There are tremendous benefits to K8S. It isn't just hype.

On the flip side, starting out as a monolithic (all in one VM) app will take significant effort to transition to micro services / K8S.

If I think I might end up at microservices/K8S, I think I might as well plan for it (abstractly) initially.