News for hackers. Stuff that matters.
Re:Facebook Workplace co-founder launches downtime fire alar... by mjayhn on Monday February 10, 2020 @02:36PM (#22292410)
Being able to take a snapshot of the chat/repo pushes/CI+CD jobs/grafanas and everything else going on during an outage and separate it by an incident type/tag/hostname/etc for later reference (to write RFOs, to more quickly solve it if it occurs again, to write documentation on, etc) is something I've wanted for awhile so this seems really interesting.

I'm sure that stuff exists (I think Datadog sort of does this, etc) but I've yet to work anywhere that doesn't just create some #shtf-$date slack channel which eventually gets lost in the black hole due to cost prohibition or time required to get it going while a fire is going on.

Re:Facebook Workplace co-founder launches downtime fire alar... by quartz on Monday February 10, 2020 @02:47PM (#22292518)
100%. Finding ways to identify the relevant data related to an incident is something that requires a lot of additional integrations that we're actively working on.

One thing Kintaba is really good at right now is wiring your slack channel directly into the activity log so it's properly attached to the incident and ultimately the postmortem. This helps avoid that channel getting lost over time, but there's still lots to do for sure in organizing all that data to make it more useful after the fact (one thing we currently support is #tagging for quick search within the log).

Re:Facebook Workplace co-founder launches downtime fire alar... by duxup on Monday February 10, 2020 @02:56PM (#22292596)
Agreed, a repo snapshot, every log you can think of, all that good stuff that you do manually to figure out if this = that.

Few things are as annoying as going back in repo time and trying to figure out if you're in quite the same place / right place.