It’s not either AWS or “bring your soldering iron”. Self-hosting K8s on managed hardware is still insanely cheaper than AWS.
GitOps, infrastructure as code, DevOps platforms, CNCF best practice setups are entirely doable without paying the Amazon tax. If your lead DevOps quits, you’re right, it’s easier to find someone who does AWS.
The last time I worked for a company that was invested in AWS, their AWS budget corresponded to five full-time senior developers in running costs.
It’s not much different than outsourcing DevOps, which I think is what some hackers frown upon, because that’s their skillset you buy in town at a premium.
> The last time I worked for a company that was invested in AWS, their AWS budget corresponded to five full-time senior developers in running costs.
This means nothing to me -- what is the TCO of the AWS deployment vs. the TCO of the non AWS deployment? How flexible are the different systems for the future? What are the reliability/uptime requirements and the marginal cost of outage? How replaceable are the dev ops engineers who are running your K8s/managed infra?
People moving to cloud isn't irrational. There's a reason that it's pervasive.
I can only speak of one PHP/TypeScript app with a poor performance profile: from $250/mo to $5.000/mo in running costs, not counting the migration. The team that made the app couldn’t maintain the Helm chart or the Terraform resources, adding to a DevOps team bottleneck that appears to limit how teams innovate (kind of how like creating new branches was expensive on CVS).
> How replaceable are the dev ops engineers who are running your K8s/managed infra?
I haven’t met a competent AWS DevOps engineer who wasn’t also competent to run the clusters; it’s just that they prefer not to.
> People moving to cloud isn't irrational. There's a reason that it's pervasive.
Yes, it’s made really solid. AWS makes sense to reduce operational risk at a large scale.
The price is not my main argument, although you’ve gotta make good money to justify the expense.
My motivation is to own one’s own infrastructure. I like to work on things that are both important and contented enough that you can’t, asa principle, outsource the operation.
Power, air-conditioning, security etc isn't my problem. That's what we pay the colo for.
I go there about once a year to install new servers, and about once at some other point in the year to replace a failed disk. I've taken various developers who are interested each time, so if I'm out of town there are several people who can cover, plus the colo staff.
You can also pay for people to do this routine work.
And then there's still the option of renting managed servers, so that company handles all the hardware.
I've found there's generally developers who are into building custom gaming PCs or Raspberry Pis or whatever who are interested to see the servers and lend a hand installing new ones.
I do not want to be on call 24/7.
You can't get much more time efficient than data centre operators handling hardware.
Now there’s NVMe and there are quite a few form factors. And NVMe is generally a point-to-point link to the CPU, so you either need a motherboard with the right number and types of connectors or you need a PCIe/NVMe splitter or switch. And the latter don’t seem widely available on the non-sketchy-hardware market.
So the situation is fine for system integrators who make their own PCBs, but not amazing for DIY builders.
I worked in an IT department as an intern. I thought this company should build custom PCs for all their workers. You can get a much faster CPU or much faster GPU for the same price! It's a no brainer!
Then I realized that if I wanted to build custom PCs for all the workers, I'd have to provide support for all of them. I'd have to be as good as CDW, who was supplying the company with PCs. This means next-day repairs and replacements. Drivers. Security. Firmware updates. For hundreds of PCs.
Needless to say, a lot of people under appreciate all the things that vendors like AWS or CDW provide. They think they can do it because they know how it's done. But they're ignoring a lot of other factors that can come back to bite you.
I'm sure you're very competent at managing server hardware. But if I'm a seed stage startup, I'd politely decline your offer.