Re:Chaos Monkey Guide for Engineers
by jedberg on Saturday February 16, 2019 @06:40PM
(#19181670)
Perhaps if you had introduced Chaos Monkey, those other problems would have gotten solved as a solution to the Chaos (tm). :)
[ Parent
| on HN ]
Re:Chaos Monkey Guide for Engineers
by karlkatzke on Saturday February 16, 2019 @10:27PM
(#19182521)
No. That’s entirely not the way it works. Chaos Engineering works in companies with significant scale and sufficient maturity, and the purpose is to discover design or implementation failures in distributed systems before they happen during big rushes where you’re hopefully making lots of profit.
If you lack sufficient maturity or you are not large enough, you will actually lose benefits if you try to implement chaos engineering. It’s like teaching a toddler to use a propane torch. You just don’t do it, or everything you love in the world will burn.
Re:Chaos Monkey Guide for Engineers
by karlkatzke on Saturday February 16, 2019 @10:39PM
(#19182549)
Self reply to realize that I shouldn’t lecture my grandma on how to suck eggs.
Maybe I should have added a disclaimer with the :) at the end.
Thanks for the laugh though. Your phrasing was awesome.
I’m telling you from experience that’s wrong. We did it at a project with just 15 or 20 developers, starting from day one of development. Literally the first thing we installed was chaos monkey before doing anything else, even before getting Jenkins running.
Re:Chaos Monkey Guide for Engineers
by delusional on Saturday February 16, 2019 @08:45PM
(#19182122)
I think it was Netflix engineer at a conference that once said that you shouldn't run chaos monkey if you expect it to fail. If you don't think it will work, then go fix those places that you know to be rotten first.
Your comment was probably in jest, but it provided a nice platform for mine :)
It was indeed in jest. And there’s a good chance I was the Netflix engineer who told you that.
To be fair though when we launched it at Netflix we knew it would break things. But we also knew we had the corporate will to deal with it.
> But we also knew we had the corporate will to deal with it.
That's really the key. If you don't have that, it's not going to work out in the long term.
Re:Chaos Monkey Guide for Engineers
by marcosdumay on Saturday February 16, 2019 @07:13PM
(#19181799)
If the GP introduced a Chaos Monkey, he would probably get fired by the same people that are not fixing those problems.