News for hackers. Stuff that matters.
Re:We reduced our server costs by moving away from AWS by dijit on Wednesday September 28, 2022 @03:30PM (#33012168)
There is a value prop for sure.

It’s hard to quantify how best you’ll be served but a lot of people are following the mantra of “nobody got fired for going AWS”.

It makes sense for some people, others are cargo-culting; yet more are fanning the flames of that cargo cult because their pay check depends on it.

Sysadmins are/were paid much less than cloud native devops people, and you need the same number of them unless you keep things very simple, which cloud providers do not incentivise. One need only look at AWS reference architectures.

Re:We reduced our server costs by moving away from AWS by icedchai on Wednesday September 28, 2022 @03:50PM (#33012372)
One good DevOps person can achieve much more than a single sysadmin. Most old school sysadmins were doing everything manually with relatively little automation.

You are certainly correct about the the overly complex AWS reference architectures. I've seen relatively simple applications with just as much infrastructure code (generally "terraform", occasionally CloudFormation JSON) as application code. It's crazy.

Re:We reduced our server costs by moving away from AWS by dijit on Wednesday September 28, 2022 @04:10PM (#33012631)
You’re quite mistaken on the first point.

Most automation that you know of as “devops tools” are borne from sysadmins.

Terraform was written by a sysadmin; puppet, ansible, saltstack and cfengine too.

It is revisionist to think sysadmins were not automating their jobs.

Old school sysadmins used to know C, bash and Perl.

New school devops just traded that for a handful of DSLs and Python.

I’m sure some technicians working the IT department managed to get by running scripts created by sysadmins and claiming not to code, but it was definitely the common case that 20 years ago sysadmins could code and worked tirelessly to “automate themselves out of a job” (literally a mantra i was told as a sysadmin 15 years ago)

Re:We reduced our server costs by moving away from AWS by icedchai on Wednesday September 28, 2022 @04:40PM (#33012953)
I'm sure we all had different experiences with old school sysadmins. The ones I'm familiar with (early 90's, ISP industry) could do shell scripting and some perl. C was way out of their skill sets, except for a bit of copy-and-paste.

The people who wrote tools like terraform and ansible were engineers with system administration skills. Those are very rare.

Re:We reduced our server costs by moving away from AWS by dijit on Wednesday September 28, 2022 @04:47PM (#33013054)
I’m biased. I worked in teams with such people basically my whole career.

The pay increased, the titles and tools changed but the mentality didn’t.

People just started beating their chest about devops and pooping on the legacy of sysadmins which is what most devops/SRE are.

Sounds like they were automating though.

“Cattle not pets” was a sysadmin mantra, but the business wanted pets most of the time.

Re:We reduced our server costs by moving away from AWS by icedchai on Wednesday September 28, 2022 @04:55PM (#33013144)
There was certainly some automation being done by the sysadmins I was familiar with (I did say "relatively little" in my original post, meaning compared to today!) It was definitely more pets than cattle at the places I was familiar with. These were small-ish companies, and the business didn't care as long as the systems worked.
Re:We reduced our server costs by moving away from AWS by dijit on Wednesday September 28, 2022 @04:57PM (#33013166)
Sounds like a “devops” in that same position would be doing the same then.
Re:We reduced our server costs by moving away from AWS by jen20 on Thursday September 29, 2022 @08:39AM (#33019733)
> Terraform was written by a sysadmin

I’m with you on most of your post, but this is straight up wrong.

Re:We reduced our server costs by moving away from AWS by dijit on Thursday September 29, 2022 @08:49AM (#33019842)
The original author of terraform (Mitchel Hashimoto) was not an Operations Engineer before writing Terraform?

https://www.linkedin.com/in/mitchellh/

Or are you saying that he's not the author? Commits seem to suggest otherwise.

  commit fb9c58f0e20a85a5358142ddb39a2f74d3bbe69b
  Author: Mitchell Hashimoto <[email protected]>
  Date:   Fri May 23 11:03:38 2014 -0700

     config: better error message

  commit 089822a36f0a5e4c583ab401b7496b48bfa31d65
  Author: Mitchell Hashimoto <[email protected]>
  Date:   Fri May 23 10:52:19 2014 -0700

     config: some comments

  commit ec3f72703c82f50787ef86498d856a5798900ad8
  Author: Mitchell Hashimoto <[email protected]>
  Date:   Thu May 22 16:56:28 2014 -0700

     Initial work on config

  commit 649cf336e86da10910f6e50ea3cfaad507a2e336
  Author: Mitchell Hashimoto <[email protected]>
  Date:   Wed May 21 16:28:53 2014 -0700

     Initial commit
Re:We reduced our server costs by moving away from AWS by jen20 on Friday September 30, 2022 @12:55PM (#33037330)
Per my understanding (as one of the earliest contributors to Terraform and employees of HashiCorp) that indeed he was a developer with an interest in operations, not a sysadmin. Mostly consulting work for rails.
Re:We reduced our server costs by moving away from AWS by arinlen on Thursday September 29, 2022 @09:00AM (#33019973)
> (...) an Operations Engineer (...)

An operations engineer and a sysadmin are two entirely different roles.

A system administrator administers systems while an operations engineer dedicated their time writing code to automate processes.

Re:We reduced our server costs by moving away from AWS by dijit on Thursday September 29, 2022 @09:04AM (#33020027)
You're conflating an IT Technician and a Sysadmin.

Sysadmins automate their work. Operations Engineer is just another title for sysadmin.

Systems Engineer is also another title for the same thing.

Platform Engineering (depending on where you are) is yet another variation.

If you're responsible for the operating system, monitoring, clustering, hardware etc; then you're firmly in "sysadmin" land.

Re:We reduced our server costs by moving away from AWS by boltzmann-brain on Thursday September 29, 2022 @12:07PM (#33022616)
This whole string of conversation is some of the most horrendous navel gazing I've had in my comments for a while.
Re:We reduced our server costs by moving away from AWS by dijit on Thursday September 29, 2022 @12:10PM (#33022667)
Welcome to Hackernews.

Pedantry is requisite.

Re:We reduced our server costs by moving away from AWS by arinlen on Thursday September 29, 2022 @01:05PM (#33023728)
> Sysadmins automate their work.

So do janitors. What's your point?

> Operations Engineer is just another title for sysadmin.

It really isn't. That makes as much sense as claiming that DevOps is just a sysadmin that's also a developer.

Re:We reduced our server costs by moving away from AWS by icedchai on Thursday September 29, 2022 @04:44PM (#33026912)
Job titles are mostly meaningless when comparing across organizations (and often within organizations, when they are large enough.)

Many "operations engineers" are actually sysadmins. Many "devops" are really sysadmins also doing development.

Re:We reduced our server costs by moving away from AWS by jjav on Wednesday September 28, 2022 @04:19PM (#33012737)
> Most old school sysadmins were doing everything manually with relatively little automation.

That's not true of any place I experienced in the early to late 90s. If you meant earlier, perhaps, I wasn't there.

The growth of perl, for example, was in great part from the sysadmin community automating everything.

Re:We reduced our server costs by moving away from AWS by icedchai on Wednesday September 28, 2022 @04:43PM (#33012989)
Most of the scripts I was familiar with from that time were one-offs. The code wasn't very reusable. They were automating a task on one machine. Today's "DevOps" are automating things across N machines. It is a matter of scale.
Re:We reduced our server costs by moving away from AWS by jjav on Wednesday September 28, 2022 @05:04PM (#33013238)
> They were automating a task on one machine.

Remember that in those days you likely only had one machine. By "you" I mean the whole department. Everyone was logged into it and it handled email, talk, documents, compilation and debugging, etc.

That doesn't take away from the fact that the sysadmins were automating everything they needed to do, in the enviornment that existed.

Re:We reduced our server costs by moving away from AWS by doctor_eval on Wednesday September 28, 2022 @06:20PM (#33013990)
Exactly - I used to be the sysadm on a single machine in a medical laboratory in the 90s. It was a DG Aviion and with the storage unit took up a whole room. Maybe 300 people used it simultaneously.

The idea that I would need to automate the installation of an OS and applications on a fleet of machines was never contemplated because it didn’t make sense. I had one machine and OS upgrades arrived in the post every 6 months or so - on cartridge tape.

It was about need, not competence.

Re:We reduced our server costs by moving away from AWS by icedchai on Wednesday September 28, 2022 @06:41PM (#33014157)
The places I'm talking about were mid 90's, dozens of AlphaServers, Sun Sparcs, HP/UX systems, IBM AIX machines, etc. It was more than a single machine, but less than 50. There was shocking little automation. Everything was a pet, with crazy NFS mounts all over the place, custom scripts in /usr/local/etc not in version control, etc. If there was ever a power loss, it was a pain getting everything back up.
Re:We reduced our server costs by moving away from AWS by dijit on Wednesday September 28, 2022 @06:50PM (#33014216)
I think we’re talking passed each other by talking different eras.

I worked in the mid-00s and it wasn’t a lot better in the dev space. People passing USB keys to each other was pretty common, SVN and CVS were around but there was a lot of developer code outside of it and peer review was going to someone’s desk and walking through the new code, nothing at all like what we have today for processing change requests.

You’re talking about systems administration when it was going from pets to cattle, the primordial period where people were automating but not applying software development practices on themselves yet because even the software development practices weren’t well defined.

Sysadmin was always lagging 5y behind development w.r.t. programming practises. The same is true today of devops.

tell me how many terraform repositories have unit tests or infra scripts for that matter which would be much simpler.

Re:We reduced our server costs by moving away from AWS by icedchai on Wednesday September 28, 2022 @07:12PM (#33014402)
Yes, I think you are right. The era I'm recounting is from roughly 1995 to 2001-ish at various local ISPs and medium size dev shops.

I did know some fantastic sysadmins at a local university around that time period. They had huge labs of Alphas and Suns and automated everything! Unfortunately, those folks didn't go into industry.

Re:We reduced our server costs by moving away from AWS by doctor_eval on Wednesday September 28, 2022 @10:54PM (#33015796)
Yes - I was talking early 90s, certainly pre-1995. There was certainly a rough period after that as the internet emerged, and I do remember being frustrated that there weren’t better tools for deploying and maintaining servers in the various ISPs I worked in. As a developer I wrote some tools to help the sysadms but it was a difficult period and I’m not sure that period has really finished yet which might be why AWS has become so popular.

The closest I’ve seen to devops nirvana has been NixOS, but its frustratingly divergent terminology and file formats make it very hard to cross the chasm.

Re:We reduced our server costs by moving away from AWS by boltzmann-brain on Thursday September 29, 2022 @12:09PM (#33022648)
Eh, nah, not exactly, back in the day people would do stuff via install scripts in Debian/RH packages, or before that, in Makefiles. You'd encounter a lot of that stuff back in the day. It always sucked. But it was there.
Re:We reduced our server costs by moving away from AWS by boltzmann-brain on Thursday September 29, 2022 @07:00AM (#33018863)
I mean, you're not wrong in that there's definitely a massive difference between the power of tools that a devops engineer can use today vs what a sys admin had access to back in the day before the devops movement was established (and that was a clear break point that was announced all over the likes of HN and there was a manifesto and all that).

But nowadays if you want to manage your own metal then you can use those tools as well. You can still run hypervisors VMs on your own servers. You're just not paying Amazon 6x the price of what it would cost you, and you're not relegated to low-performing compute, memory, and IO elements that Amazon standardized on.

Re:We reduced our server costs by moving away from AWS by Melatonic on Wednesday September 28, 2022 @04:16PM (#33012703)
Not necessarily true - many were doing tons of scripting.