News for hackers. Stuff that matters.
Re:Elon Musk is disconnecting random Twitter-servers just to... by sacnoradhq on Sunday December 25, 2022 @07:24AM (#34126635)
To a non-engineering manager or non-SRE, this may seem absurd.

We do these in a more planned way where I work and call them "fire drills".

Everything prod should be HA and evict-able from a particular server, rack, and/or DC floor and survive. If it's not, it's a sign something is wrong.

The goal isn't fun, it's to find corner cases and places that aren't HA by meatcloud, chaos monkey-style.

Ideally, lights-out PDUs and Ethernet switch port control can simulate "pulling plugs" in an automated way and do so remotely and more cheaply.

If you bought a company, maybe you'd do this once to get a baseline, but with the express intention of not doing it for fun or to be a jerk.

There are no comments on this comment yet.