While it of course won't work in every case, in this case Slack wasn't down, you just couldn't attach files in some situations. This is something Slack easily could have communicated as soon as you tried to attach a file. Now it look like it worked and then times out after a couple of minutes and deleted your message
Ever since this started, been seeing 100% failure on webhooks for Event subscriptions.
But in this case Slack was not down. Parts of the service were frustratingly lagging, like uploading a picture or using reactions. Circle CI does have this sort of warning on their sidebar. There's so many parts to a big software product that it would definitely useful to know if some part of it was having issues.
If it's on a different domain and servers, I don't think it will be an issue.
If they have it on an API call to (making it up) api.slack.com/status it might very well go down when things go wrong, making it useless.
However if they physically host it elsewhere ideally on a different cloud provider, with a different domain, say, slackstatus.com/api-status that is completely separate from the actual services, which can both query the actual service with a call from outside and can be updated manually with physical persons at Slack, and the client checks that domain, I think it can work.
this sounds very complicated lol
Re:Incident at Slack
by sharklazer on Wednesday March 09, 2022 @09:11AM
(#30614543)
No, not really. If your calls time out, ping the status server. Done.
It sounds more like laziness than complication to me. But that typifies Slack dev culture—“just use duct tape”
And that's exactly how you DDOS your status server.
Re:Incident at Slack
by matthew-wegner on Wednesday March 09, 2022 @10:28AM
(#30615499)
Why can't they put a status.json file on S3/CloudFront/R2/wherever?
And in an absolute worst-case infra collapse, someone there could manually edit/upload a new one...
Re:Incident at Slack
by charcircuit on Wednesday March 09, 2022 @03:44PM
(#30619718)
Put a CDN in front of the status server. If that overwhelms your CDN then add some exponential back off to the mix.
When Slack is down, Slack can't tell you when it is down. It's why status pages are often on other domains/systems etc, so the status page remains up while everything else is on fire.