News for hackers. Stuff that matters.
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by awslol on Thursday November 16, 2023 @05:30PM (#38296550)
They also saved the salaries of the team whose job was doing nothing but chasing misplaced spaces in yaml configuration files. Cloud infrastructure doesn't just appear out of thin air. You have to hire people to describe what you want to do. And with the complexity mess we're in today it's not at all clear which takes more effort.
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by gymbeaux on Thursday November 16, 2023 @05:41PM (#38296684)
Exactly. Last job I worked at there was always an issue with the YAML… and as a “mere” software engineer, I had to wait for offshore DevOps to fix, but that’s another issue.
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by scarface_74 on Thursday November 16, 2023 @07:25PM (#38297626)
If you are waiting on a “DevOps department”, it isn’t a DevOps…it’s operations
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by gymbeaux on Monday November 20, 2023 @02:53PM (#38353558)
My company called them the DevOps Team
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by icedchai on Thursday November 16, 2023 @07:51PM (#38297825)
Did you try and fix it yourself? Was that not allowed?
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by gymbeaux on Monday November 20, 2023 @02:52PM (#38353539)
Yeah that was not allowed. We started off managing our own DevOps in our own AWS region even, separate from the rest of the company. Eventually the DevOps team mandated we move to the same region as everyone else, and then soon after that there was enough conflict with my (offshore) team and DevOps (onshore) that DevOps demanded and got complete control over our DevOps, though not infrastructure.
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by quickthrower2 on Thursday November 16, 2023 @05:41PM (#38296680)
100% this. Cloud is a hard slog too. A different slog though. We spend a lot of time chasing Azure deprecations. They are closing down a type of MySQL instance for example for one which is more “modern” but from the end user point of view it is still a MySQL server!
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by icedchai on Thursday November 16, 2023 @05:42PM (#38296693)
To manage a large fleet of physical servers, you need similar ops skills. You're not going to configure all those systems by hand, are you?
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by postalrat on Thursday November 16, 2023 @06:32PM (#38297165)
They spent $150,000 on physical servers. Probably 1 or 2 racks. Not much of a 'fleet'.
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by ketzo on Thursday November 16, 2023 @07:18PM (#38297570)
I mean, “fleet” semantics aside, surely $150,000 of servers is enough for at least one full-time person to be maintaining them. The point is that there are absolutely maintenance and ops costs associated with these servers.
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by dijit on Thursday November 16, 2023 @07:25PM (#38297622)
There's maintenance cost to everything that runs a userland.

What we're missing is tracking what time is spent managing hardware and firmware, (or even network config, if we're being generous) and how much time is being spent on OS config.

From personal experience (as a sysadmin before it was entirely unsexy as a term) the overwhelming majority of my ops work was done in userland on the machine, maybe something like 96-97% of my tasks were nothing to do with hardware at all.

Since I got rebranded as SRE, the tools and the pay sure did get a lot better in the time, but the job is largely similar and ultimately running in VMs does make deployment faster, but once deployed I find the maintenance burden to be the same (or perhaps a little more) as things seem to become deprecated or require changes from our cloud vendor a bit more often.

Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by awslol on Thursday November 16, 2023 @05:59PM (#38296869)
Depends on the size of the fleet.

If you're using less than a dozen servers manual configuration is simpler. Depending on what you're doing that could mean serving a hundred million customers. Which is plenty for most business.

Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by icedchai on Thursday November 16, 2023 @06:09PM (#38296975)
A dozen servers would be pushing it. It's not the size of the fleet, it's the consistency and repeatability of configuration and deployment. I assume there are other, non-production servers: various dev environments, test/QA. How do you ensure every environment is built to spec without some level of automation? It's the old "pets" vs "cattle" argument.

I've worked at companies with their own data centers and manual configuration. Every system was a pet.

Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by notnmeyer on Thursday November 16, 2023 @06:06PM (#38296951)
sorry, what? theyre running on k8s and using helm... so there's still piles of yaml. its wild to conflate migrating to bare metal with eliminating yaml-centric configuration.
Re:Moving from AWS to Bare-Metal saved us 230k$ /yr by PartiallyTyped on Thursday November 16, 2023 @11:57PM (#38299750)
CDK exists. Haven’t had to use yaml in ages and I refuse to.