Re:Finding a problem at the bottom of the Google stack
by alephnan on Sunday March 15, 2020 @12:03PM
(#22584152)
> We consider these types of failures "within error budget" events. They are expected, accepted, and engineered into the design criteria of our systems. However, they still get tracked down to make sure they aren’t forgotten and accumulated into technical debt
[ Parent
| on HN ]
Re:Finding a problem at the bottom of the Google stack
by Nextgrid on Sunday March 15, 2020 @12:07PM
(#22584180)
Yes, I understand that application-level exceptions can sometimes be "budgeted for" and a certain amount of them is acceptable.
Hardware metrics on the other hand should IMO be monitored and alerts should be in place for when things go wrong, as the consequences of it could potentially be way more serious.
Re:Finding a problem at the bottom of the Google stack
by jefftk on Sunday March 15, 2020 @09:06PM
(#22588573)
Things look very different when you have as much hardware as Google does.
(Disclosure: I work for Google)