Not sure why you get the downvotes, all I can imagine is that there are a lot of people here who learned on K8s and have never tried anything different, or had to manage infrastructure at scale (In my experience scale being 30k+ bare metal machines)

99.9% of the time I see a k8s implementation it's a shop pushing a ridiculously low amount of traffic and then needing to scale their application across decades-old hardware provided by a cloud provider. It's so bad that a single, modern 4U server, like I used to manage working at at a large company, could have replaced their entire infrastructure. The k8s users wind up with thousands of lines of YAML to solve a problem that could have been solved with better design decisions. The abstractions upon abstractions also prevent developers from truly understanding what is going on in the real world, for example your cloud provider's hypervisor doesn't align with your para-virtualization hypervisor so you end up with all sorts of issues with affinity and noisy neighbors on the same hardware that you can't even see, or your racks aren't splayed right so failures wipe out disproportionate amount of instances, that K8s then takes ages to rebalance things.

As far as the statement "there's no reason it should be as simple as Heroku", I agree completely. I imagine in a few years something like AWS Firecracker+ a simpler control plane will end up superseding k8s, and k8s will end up in the graveyard of dying virtualization orchestrators like OpenStack and CloudStack.

> I imagine in a few years something like AWS Firecracker+ a simpler control plane will end up superseding k8s, and k8s will end up in the graveyard of dying virtualization orchestrators like OpenStack and CloudStack.

Past experience makes me pessimistic that we will ever get something "easy" that everyone uses.

Doing all of that is my job and you are 1000% correct. I will always have job security now because of how ridiculously over-complicated, clunky, rigid, flaky, and bizarre all the DevOps tools are. K8s Admin is now the new DBA. Writing Terraform is a programming job in itself. It's like somebody decided we don't have enough useless people in tech so they made all this new tech be more annoying than it has to be. We could probably replace it all with a couple shell scripts and it'd work just as well.

To add to that, I find very few engineers actually "writing" anything in these IaC languages. It's like a big game of copy/paste where people simply keep repeating bad patterns that use a library that uses an API they don't understand, maintained by someone working at a cloud provider who barely understands how their own service works. Like, there are so many turtles all the way down, when you see this happen long enough it drives you crazy.

I am like that. To get new applications working in my company, I have to run a proprietary shell script that creates about 20 terraform files that I then later have to manually adapt via copy and pasting. Then I have to enter several configs/settings into our config server, etc.

In the first company I ever worked for, we could run one-click jobs that would build and deploy .war packed applications to our tomcats instead. They were based on 1) building the application with mvn and 2) pushing the .war to the tomcats. Each version was running in parallel, the version agnostic path routing could to the relevant version could be set via dropdown.

> I have to run a proprietary shell script that creates about 20 terraform files that I then later have to manually adapt via copy and pasting.

Would it be possible to replace this with modules and variables? I've done this a few times although my scale is incredibly small at the moment.

Yes and no. There's a diminishing return after a certain amount of modules and variables. It's because the tool is shitty and limited. For example there's no such thing as inherited or global variables between Terraform modules, you have to explicitly pass them, or else pass these clunky object variables that embed lots of data and types but not the actual variable concept. And at the end of the day you need one final root module which requires a bunch of files, so you might as well use a template.

Thanks for the self awareness. I do not hate devops engineers, I am fully onboard with engineering for job security, but I am against anonymously pretending it's anything but.

As soon as we brought our redis and mongo in house, k8s became easier and cheaper than heroku (and 2 other PAAS solutions that we had experimented with).

After the switch (and a single open source tool) we still didn't need any specialized devops knowledge, we just have developers in house.

A lot of easy things suddenly become very complex if you need to deal with high availability and/or a lot of distributed computers.

Like counting distributed streaming data with high accuracy and low latency - or especially calculating quantiles - for example.

Not at all. I worked on HA and distributed systems for 10+ years and the best distibuted designs are by far the simplest.

Why are you so bitter about devops people?

I am not bitter about them, personally. If I could pick one area to work in, in the field of the technical side of software engineering I would go for devops. I applaud everyone who earns their money with this dreck!

But taking a step back, it's madness. I have seen mindbogglingly complex setups 10 years ago, when I had to go begging to the old websphere greybeards when I needed anything sysadmin/devops related, yet in that time it has gotten worse. 4 of the 4 kubernetes projects I have seen personally involved highly customized code for deployment and infrastructure, that wasn't comprehensible for anyone besides the relevant devops teams. From the perspective of efficiency and the company owner(s) it appeared borderline malicious.