News for hackers. Stuff that matters.

A comment on Root cause analysis: significantly elevated error rates on 2019‑07‑10. Parent comment

Re:Root cause analysis: significantly elevated error rates o... by londons_explore on Friday July 12, 2019 @07:06PM (#20425315)
> My concern here is, a deployment have been made months ago, and many other changes that could make things worse were introduced.

Make every bit of software in your stack export as a monitoring metric it's build date. Have an alert if any bit of software goes over 1 month old. Manually or automatically re-build and redeploy that software.

That prevents 'bit rot' meaning you daren't reduild or rollback something.

[ Parent | on HN ]
Re:Root cause analysis: significantly elevated error rates o... by bdamm on Saturday July 13, 2019 @05:26PM (#20430264)
In a lot of environments this is a terrible idea. In private environments exposing build manifest information is a good idea, but not so that you can alert at 1 month. Where I work, software that's 2-3 years old is considered good - mature, tested, thoroughly operationalized, and understood by all who need to interact with it on a daily basis. Often, consistency of the user experience is better than being bug free.
[ Parent | on HN ]