News for hackers. Stuff that matters.
Re:How to Build Software Like an SRE by throwaway012282 on Monday October 17, 2022 @06:04AM (#33231555)
Sorry for the throwaway.

> I’ve been doing this “reliability” stuff for a little while now (~5 years), at companies ranging from about 20 developers to over 2,000

I've been doing it for 20+ years including running critical services in FAANGs.

> Use Docker > Use Kubernetes.

Hard pass. There's a reason why most FAANGs developed their own packages management and deployment system and keep using them. They are simpler, less bloated, easier to debug.

> Deploy everything all the time

On your local testbed, maybe, unless you already know a commit is broken. On production, absolutely not.

Re:How to Build Software Like an SRE by adamckay on Monday October 17, 2022 @07:16AM (#33232034)
> There's a reason why most FAANGs developed their own packages management and deployment system and keep using them. They are simpler, less bloated, easier to debug.

And for every other company that isn't one of those 5, rolling your own deployment system won't be simpler, leaner or easier to debug. Nor will it, in all likelihood, be as stable, feature rich, observable, etc.

Re:How to Build Software Like an SRE by greymalik on Monday October 17, 2022 @07:32AM (#33232147)
> There's a reason why most FAANGs developed their own packages management and deployment system and keep using them. They are simpler, less bloated, easier to debug.

FAANG-appropriate solutions aren’t always appropriate for the rest of the world.

Re:How to Build Software Like an SRE by evercast on Monday October 17, 2022 @07:37AM (#33232193)
> There's a reason why most FAANGs developed their own packages management and deployment system and keep using them. They are simpler, less bloated, easier to debug.

Most FAANGs developed their computing infrastructure before Kubernetes gained popularity. After all the investment into building it and fine tuning for their software services (like ads), they probably have a lot more reasons to not migrate to something different. Not just because their systems are "simpler, less bloated, easier to debug".

Re:How to Build Software Like an SRE by nine_k on Monday October 17, 2022 @09:33AM (#33233216)
Interestingly, it's G in FAANG who came up with K8s in the first place.
Re:How to Build Software Like an SRE by keyle on Monday October 17, 2022 @08:19AM (#33232519)
I'm nowhere near faangs but I agree with your view, coming from about 20+ years of development.
Re:How to Build Software Like an SRE by tikkabhuna on Monday October 17, 2022 @09:09AM (#33232940)
> Hard pass. There's a reason why most FAANGs developed their own packages management and deployment system and keep using them. They are simpler, less bloated, easier to debug.

That seems like an odd thing to say when Kubernetes came out of Google and they based it on Borg, their internal orchestrator.

FAANG create many things themselves as they're trend setters. They have the size to create them. Having worked on in-house solutions, it is incredibly difficult to maintain adequate documentation, training, and resources to keep it going.

Re:How to Build Software Like an SRE by nijave on Monday October 17, 2022 @09:35AM (#33233247)
> Deploy everything all the time

More often than not, this ends up being a business decision about balancing risk with potential loss of revenue (you don't want to take your stuff down, but you also don't want to delay new features that could bring more business)

Fast growing companies trying to enter the market tend to favor "deploy more often, break more often" while companies with large, established bases tend to favor "more testing, more deliberate changes"

I would say the inverse is a better rule "you should be /able/ to deploy any time"

Re:How to Build Software Like an SRE by nine_k on Monday October 17, 2022 @09:37AM (#33233273)
Why, deploying everything all the time is fine, it's what FB, for instance, does. If you don't, there is a hazard of accumulating too much change per deployment, which becomes more dangerous to deploy, and more problematic to roll back.

But it does not mean "deploy every change to 100% of the fleet instantly". Use canaries, use slow, monitored rollout. With smaller changes, it's relatively painless and routine.