Kubernetes is much more simple than what we would have to do without it, and my team is much much smaller than anything at Google. For what it does, it offers some good opinions for what might otherwise be a tangle of dev ops scripts.

If what you want to deploy is best described as “an application” it’s probably not the right tool for the job. If what you want to deploy is best described as “50 interconnected applications” it’s probably going to save you time.

> If what you want to deploy is best described as “an application” it’s probably not the right tool for the job. If what you want to deploy is best described as “50 interconnected applications” it’s probably going to save you time.

This is an excellent way of looking at it. I've struggled for many years to come up with a response to hacker news comments saying you don't need kubernetes, but this sums it up about as well as I could imagine.

As someone who runs both in production, Nomad would almost certainly meet your needs.

Learning how to operate Kubernetes well takes a while and I would say is only worth the investment for a extremely tiny percentage of companies.

Maybe so, but anyone should definitely use more criteria than my few word generalization to choose their deployment infrastructure. :)

We (mostly) chose k8s over other solutions because of other tools/providers in the ecosystem that made business sense for us. But we did need something to abstract our deployment complexity.

I’m mostly suggesting that I suspect many of the people with bad k8s experience didn’t really need it.

I’ve seen a number of people wrap a simple application in a container, slap it in a deployment/service/ingress and call it a day, it works, but using k8s that way doesn’t really add much value.

Maybe, but for better or worse it's also become the industry standard, much like Terraform.

If you don't know k8s and Terraform, you're shooting yourself in the foot for future jobs.

K8s is an enormously complex piece of software and I haven't met a great many people who "know" it inside and out.

Basic concepts and how to write a job/service/ingress, sure. Knowing the internals and how to operate it? I'd say that's only for specialists. Most people don't need to know what a Finalizer is or does. Most people aren't going to write operators.

It is a multi-year investment of time to deeply understand this tool and it's not necessary for everyone.

The same could be said for the linux kernel, and yet we still run all of our software on it.

Except with the kernel, you only have to be familiar with the system calls and you don't need a team of people just to run, maintain and upgrade the kernel.

That and it tries to make breaking changes on the timescale of decades rather than every other minor release (so, once or twice a year?).

> Except with the kernel, you only have to be familiar with the system calls

I think it's safe to assume that any non-trivial use of linux involves non-default configuration.

> you don't need a team of people just to run, maintain and upgrade the kernel.

My relatively small company employed linux admins before we adopted (on prem) kubernetes. Their work has changed a bit since since then, but it isn't meaningfully more laborious.

I assume that less effort is required for cloud kubernetes offerings.

My whole point is that they're not really comparable from a level of effort perspective, despite claims.

Hosted Kubernetes isn't significantly easier either, as every host is offering you different things as "Kubernetes" and has different ways that you will need to manually intervene to overcome problems.

I'm only telling you this from experience, being years down the rabbit hole already.

I also speak from experience, from an organization that has had a lot of success with kubernetes. Perhaps we're in the sweet spot where our workload is suited for it but there still isn't a huge amount of complexity in maintaining it.

Agreed. I think overtime we'll just get more abstracted away from it. GKE Autopilot, for example.

I think you still have to understand the lego block in your hand though, so you can combine it well with the other parts of your system.

Modern istio provides a lot of value to a single application. mTLS security, telemetry, circuit breaking, canary deployments, and better external authentication and authorization. I’ve seen each done so many different ways. Nice to do it once at the mesh layer and have it be done for everything inside the cluster.

This is getting downvoted for cynicism maybe, but I feel it's the most important advice here. Know /when/ to use Kubernetes.

It's very often the wrong tool to deploy our tiny app but many of us go along with it because it ticks some management boxes for various buzzwords, compliance, hipness, or whatever. Once you get out this hammer factory, it's a big and complicated one, so you will probably need a full time team to understand it and manage it. It's also a metric hammer factory, so you'll need to adapt all your other tooling to interoperate. Most of us can get by with lesser hammer factories, even k3s is less management.

If you just need to deploy some containers, think hard if you want to buy the whole tool factory or just a hammer.

This kind of comment is on every single HN post about Kubernetes and is tiresome. I also think it's off topic (TFA is about kubectl tricks, not about the merits of K8s).

I think it's important to have comments like those as Google, who does not use Kubernetes, is exerting a lot of pressure on the industry to adopt it. It is an extremely complicated tool to learn to use well and companies act like there aren't reasonable alternatives.

Those of us who have gone through it are often coming back with war stories saying to use something else. Some of us have invested thousands of man hours into this already and have strong opinions. At the very least, give Nomad a look. It is maybe a tenth of the effort to run for exactly the features most people want and then some.

People need to be made aware that there are options. I have friends at companies that have large teams just dedicated to managing Kubernetes and they still deal with failure frequently or they spend their entire day-to-day tuning etcd.

We get paid because we know these tools. It's why we're desired: because the company thinks they want K8s or they're one foot in EKS and they're doubling down. We don't get hired because we dare to suggest they dismantle their pilot cluster and take a sharp turn into Nomad.

Most of us aren't the engineering heads of our departments. So you'll forgive us if we continue pushing the moneymakers we have in our heads and setting up our homelab clusters. I want to be paid, I want to be paid well. It may as well be pushing the technology stack that scales to megacorps because who knows maybe I'll make it there one day.