You've posed a false dichotomoy. I've worked at multiple billion dollar startups that got by just fine without k8s. And I've worked at startups that started on completely different stacks and migrated to k8s. If you're already containerized, you've reserved a good amount of optionality.

The kind of reasoning you pose is similar to "SQL doesn't scale to infinity, so start with an infinitely scalable eventually consistent document store". This line of thinking is dangerous to most companies, and often the death of startups. Assume YAGNI.

There's some truth to "Nobody ever got fired for choosing IBM", but IBM was never a good choice for anyone except consultants. k8s isn't too far from that.

But you are posing a similar false dichotomy. k8s isn't some massive intractable beast that needs an army of consultants to run and maintain. It can run small loads just fine as well.

Anyone can leann k8s basic principles in a day or 2. And managing a cluster is arguably easier than managing a typical bunch of EX2 instances+ALB etc that almost everyone uses.

k8s also has the advantage of ubiquity. If I had to manage a bunch of containers on a cluster with > 10 nodes, I'd pick k8s over swarm/nomad etc.

Your argument is 'k8s isn't a good choice for anyone except Google-scale' and I don't think thats true at all.

> k8s isn't some massive intractable beast that needs an army of consultants to run and maintain.

As someone who has actively avoided k8s (despite enjoying devops), my observation has been quite the opposite. At my current (small) org we have one devops guy who spends all day working on k8s related work (configuration, troubleshooting, automation, etc.) And we're about to start looking for another devops engineer because he's swamped. We only have a handful of clusters up and running so far.

k8s is the poster child for overcomplicating matters for folks that don't have Google-scale problems.

You might be doing it wrong. I use k8s for my personal projects and the maintenance overhead is lighter than what it cost me to log into my VPS every 3 months and renew the Let's Encrypt certificate. (Which was literally just pressing the up arrow and enter, because it was always the last thing in my shell history.) At work, a team of 5 people maintain our production Kubernetes clusters (that host our cloud product and app), and develop the app itself. Which by the way, creates Kubernetes clusters for individual customers, of which we have created thousands of. Overall, it's not that hard and we don't spend much time managing computers. (We do have all of the good stuff, like all K8s changes in Git, one-click deploys, etc. I did spend a little time making Shipit run in Kubernetes, but it certainly wasn't my full time job for any extended period of time.)

There are problems that people run into. One is not needing more than one computer -- if everything you need to run fits on one computer, you might as well use systemd as your orchestrator. (Which people also say is overly complicated, but really it's just extremely misunderstood.) The other is not understanding Kubernetes enough. A lot of people just explode random helm charts into their cluster and are mad that it's hard to maintain. Basically, adding another abstraction layer only helps if you can debug the actual thing on the surface -- otherwise, you're just setting a time bomb for yourself. (And, helm charts require you beg the maintainers for every feature you could possibly need, and once enough people have done that you just have raw k8s manifests where everything is called some random word instead of its actual thing. Really bad. Use Kustomize or jsonnet!)

Compared to something proprietary like AWS ECS, K8s seems like it's almost impenetrable to learn. If you're already on one of the cloud providers, likely they will have something else to offer.

ECS isn't exactly simple either, but it is manageable for small shops where the developer is the devops guy.

it’s not only suitable for google scale problems, rather it isn’t for all problems. Like a lot of new shiny tech, it takes time investment to find out if something really is the solution to a problem. Don’t just jump in with both feet.

As one of the mentioned consultants, I think the statement that one can learn Kubernetes in a day or two grossly unfair. It is a complicated beast, for better and for worse. The (almost) unique selling point is that it is an orchestration layer that is pretty much provider agnostic in a believable way.

I didn't mean to imply you can learn it in a few days, but you can certainly learn the basic design principles and overall approach, and its pretty consistent across all the k8s products.

And that is a sound solution if you have limitless VC bux to burn while you implement your over-engineered solution, but most shops have to be a little bit scrappier.

Not to mention that you end up scaling to a size where k8s is a sound solution, that "costly" migration is going to be a rounding error on your balance sheet

> that "costly" migration is going to be a rounding error on your balance sheet

This. Just because your services are containerized doesn't mean they'll scale well with each order of magnitude. Likely you'll need to re-architect other parts of the system to work at your new scale.

> smaller than k8s is that they have a growth ceiling

Kubernetes has a serious growth ceiling - way below what Nomad has. So if you actually have a problem of interesting scale, it's almost certainly "pick Nomad now, or get blamed for a costly migration later when you realise that Kubernetes is not for you".