News for hackers. Stuff that matters.

Ask HN: What makes SRE great compared to "plain" DevOps?

In my view it is having a dedicated team focusing their full mental bandwidth on pro-actively understanding and managing robustness of the system.

In Pure DevOps, it seems to me developers often don't have the full picture of the system, and not enough bandwidth to foresee complex interactions from their changes.

These are from my experiences spending one year as a developer in somewhat large a greenfield development project for a machine learning infrastructure based on Kubernetes, where subtle platform errors would often cascade into the applications layers, causing really hard to debug problems that often left more junior application developers scramble to even know where to start debugging.

I'm preparing a presentation though, and I'm eager to hear your experiences and perspectives on this!

(SRE = Site Reliability Engineering - https://sre.google/ )

( 16 comments | on HN )
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by gtirloni on Wednesday November 01, 2023 @09:31PM (#38107830)
SRE and DevOps teams overlap a lot, with SRE being a specific implementation of DevOps, more focused on reliability and scalability by applying software engineering principles.

Sometimes traditional sysadmins will be labeled as DevOps because they are using some config management tool, but the practices are the same.

The next level is Platform Engineering, which creates a platform that encapsulates all these best practices.

You can have dedicated or distributed DevOps and SRE teams. You can't have distributed Platform team.

Re:Ask HN: What makes SRE great compared to "plain" DevOps? by lgkk on Thursday November 02, 2023 @03:40AM (#38110124)
My latest job title was platform engineer. Honestly it felt like a hack of all trades role. I was doing everything from k8s tf etc deployments observability monitoring to writing backend services running analytical queries and building UIs while managing their entire lifecycle. Really fun, enjoyed the heck out of it because every day I was learning and doing something new and interesting. 11/10 would recommend. Not sure if that’s what platform engineering is generally considered to be.
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by gtirloni on Thursday November 02, 2023 @12:31PM (#38116053)
I think that's it but what matters is if your team was exposing a platform that made the hugely complex infrastructure more amenable to developers, while helping people to stay on the golden paths of reliability, monitoring, etc. It also helps the organization to enforce some standards centrally.

What happens behind that interface can be chaotic for sure.

Re:Ask HN: What makes SRE great compared to "plain" DevOps? by toomuchtodo on Wednesday November 01, 2023 @11:47PM (#38108774)
Comprehensive knowledge of the application codebase(es) you’re responsible for and the ability to quickly determine root cause of failures and revert or remediate, either by shipping hot fixes, infrastructure changes, or both.
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by samuell on Thursday November 02, 2023 @04:38AM (#38110560)
Yea, this is at the heart of what I also thought were the very point of it.

In my experience on the mentioned project, I felt one of the few ways to get there, unless you wrote the whole codebase, is to have enough time to build (or configure existing) some tools to be able to visually understand and/or navigate the system.

During one week of being on-call engineer, inspired by the ideas of pro-activity and monitoring in SRE, I stole myself some hours to put together a script that would parse our ArgoCD configs to create a map of our Kafka topics, into a GraphViz-based HTML file with links to to the relevant topics as well as microservices (pointing to some web based inspection tools for these).

This finally made me be able to figure out what was going on and be useful during incidents as a newcomer in the field, but no one in the team had thought of this idea of being a little pro-active, so most of the time when errors occurred, people would just scramble around all over the place, and only the really senior guys would be useful because of having the whole system in their head anyway.

To me, this idea of pro-activity is what SRE brings to the table.

Re:Ask HN: What makes SRE great compared to "plain" DevOps? by hnthrowaway0315 on Thursday November 02, 2023 @08:11PM (#38122349)
Doesn't that mean it's essentially a senior developer in the dev team?
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by toomuchtodo on Thursday November 02, 2023 @08:18PM (#38122435)
It does, but a senior developer in a pager rotation.
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by hnthrowaway0315 on Thursday November 02, 2023 @09:07PM (#38122982)
It makes sense. But I do feel seniors will be pigeon holed into taking an Ops responsibility.
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by toomuchtodo on Thursday November 02, 2023 @09:12PM (#38123020)
Depends on the org. In an org where devs are protected, DevOps/infra team will do the pager duty suffering in an SRE capacity. In other orgs, seniors will be forced into it unless they have leverage to bounce otherwise, or they tolerate it because there are worse jobs out there. Culture and appetite on both sides is highly variable, won’t know until you’re on the inside.
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by VirusNewbie on Friday November 03, 2023 @03:39PM (#38133974)
It definitely can be, but depending on the application, it might make sense to demarcate 'scale/infra' related senior developer responsibilities from 'business logic' type responsibilities. An SRE would care about the former, much less the latter.

I'm a SWE SRE, and I don't particularly care to look at the code of individual features being added unless there's potential for a large blast radius, but I do care to make sure if something breaks, there isn't cascading failures. I want things to degrade in a predictable way, or to fail in an isolated fashion. The non SRE SWEs are free to break a single feature, and i'm happy to help them roll it back, but that's not what I'm primarily worried about.

Conversely, many of them aren't all that interested in thinking about how slowdowns or failures propagate upstream or downstream to other teams that may or may not understand how things are connected, whereas I do worry about that.

Re:Ask HN: What makes SRE great compared to "plain" DevOps? by rcbdev on Thursday November 02, 2023 @03:18AM (#38109986)
They'll all be washing dishes when the next recession hits IT and the need for affordability exceeds the need for reliability in service.

When the first company relearns that on-prem with dedicated admins is infinitely cheaper than cloud with similar reliability and less / cheaper staff overhead it will be a rude awakening. This is especially sad to me as most women in our field are in DevOps/SRE roles as opposed to SWE.

Pray that Bezos and co. can keep their customers / your employers fooled.

Re:Ask HN: What makes SRE great compared to "plain" DevOps? by samuell on Thursday November 02, 2023 @04:31AM (#38110520)
Though I understood one idea with SRE is that they are much more of a developer role anyway, and should theoretically have an easier time switching to a dev role if need be?
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by gardenhedge on Thursday November 02, 2023 @07:35AM (#38111967)
How is the role related to where it is deployed? Won't this role carry over to on prem deployments?
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by mathverse on Thursday November 02, 2023 @01:36PM (#38117196)
Biggest load of horsecrap I have seen on this website. Complexity ballooned into unimaginable proportions. Observability is crucial for complex software.
Re:Ask HN: What makes SRE great compared to "plain" DevOps? by not_me_ever on Thursday November 02, 2023 @07:57AM (#38112246)
What makes a great apple compared to "plain" pile of poo?

One is a role, the other is a broken mindset.

Re:Ask HN: What makes SRE great compared to "plain" DevOps? by chitraa on Friday November 10, 2023 @12:22AM (#38215354)
SRE brings a laser focus to system reliability, ensuring proactive management and a comprehensive understanding of the system's robustness.

In comparison, 'plain' DevOps might sometimes lack that specialized attention, potentially leaving blind spots in foreseeing complex system interactions. SRE is all about that extra layer of assurance for smoother, more resilient operations!