Compare https://en.wikipedia.org/wiki/Crash-only_software and https://lwn.net/Articles/191059/

In a previous job we actually had to make a decision like this: our server software was generally stable enough, but we reset it whenever we did an update. Every once in a while it would break when running for long enough.

I opted for forcefully restarting the software every day, because I'd rather relive the pains of restarting, than to discover in production what it is like for the first time to run the system for the 501st day of uptime.

(To be honest, the system in question was mostly written in Erlang, so controlled restarts were already baked into its philosophy.)

I would assume said code should never be part of actual deployments or only part of unit tests, maybe an external project even.

Oh you, sweet summer child...

Totally. I have added wifi power outlets to installations because process restart, OS restart, and hardware restart did not get a system back up and running. Yes it was a startup, and yes it was remotely installed hardware.

Might be a bit strange as you are saying this is a example of bad planning but my employer was discussing this today. We had a hard time finding options, what did you end up deploying or did you develop your own in house?

You want to search for "PDU remote management"

Edit: Another term is Switched PDU

When I had this issue I just bought some cheap wifi 'smart sockets' which had some open-source firmware available and reflashed them. They even support reporting the voltage and current draw from the socket (though not particularly accurately) and toggling the power through a POST (with a password if set). Was very useful for power cycling various bits of hardware remotely.

Make it hard to accidentally call and hide it in the docs somewhere with items related to testing.

> Make it hard to accidentally call and hide it in the docs somewhere with items related to testing.

This would be more effective if so many other AWS features weren't hidden in non-obvious places in the docs.

Make these functions only work in us-chaos-1

Make it expensive.

I'd imagine there is only a comparatively small (but earnest) pool of people who would want such an offering.

On top of keeping people who don't want it out, you'd need it to be expensive to offset the lower utilization.

I'd be okay with a garbage-tier, too. It's not failing on purpose, but only because no one gives a shit. Maybe that's just standard AWS?