Re:Communicating to Users During Incidents
by jrockway on Monday January 17, 2022 @07:03PM
(#29973160)
I've always gotten pushback because I'm told that Sales won't be able to sell if there's publicly acknowledged downtime. I'm not sure what they think 99.95% in the SLA means, if not downtime occasionally. But, that's the reality. Customers apparently compare status pages, and pick the one that claims the least amount of downtime. (I bet you can fabricate benchmarks too. People love those.)
If I ever start my own company, I'm just going to make the Grafana instance public, perhaps with the Y axis scale removed. Then users can see "hmm, we're getting a high error rate", go look at the dashboard, and say "oh, everyone is getting a high error rate". The problem I always run into as a user of a service is "did I create this bug, or is it just something they're going to fix". 99% of the time, it's something the service provider is going to fix, and they're already aware. But because they don't communicate, I have to escalate and mitigate. That's half a day gone every time. It is infuriating. (The worst part is the time when it IS your fault, and you're used to making it the provider's fault. Resolution times can be pretty low when you're playing the undercommunications waiting game.)
I tend to monitor my vendors in great detail, so I'm usually aware of an outage before they are. Nobody that I pay for a plan has any way for me to make my monitoring available to their teams. I had a call with someone once about it (a popular build-your-website-from-a-git-repo service), and they said "oh, we'd let you into our Slack for 5 figures a year". No thanks! Give me a webhook and I can ping you within seconds of your global load balancer going down for the 10th time this year.
Anyway this has become a rant, but fuck undercommunication. It wastes my day, and I am already locked into a contract, so my business isn't going anywhere. Let's be honest now that you have my money!
Re:Communicating to Users During Incidents
by rozenmd on Tuesday January 18, 2022 @01:40AM
(#29975904)
That's a great rant!
I mainly monitor my vendors since I sell an uptime monitoring tool, and it's a form of dogfooding - didn't realise there were other folks that did it too.
If I ever start my own company, I'm just going to make the Grafana instance public, perhaps with the Y axis scale removed. Then users can see "hmm, we're getting a high error rate", go look at the dashboard, and say "oh, everyone is getting a high error rate". The problem I always run into as a user of a service is "did I create this bug, or is it just something they're going to fix". 99% of the time, it's something the service provider is going to fix, and they're already aware. But because they don't communicate, I have to escalate and mitigate. That's half a day gone every time. It is infuriating. (The worst part is the time when it IS your fault, and you're used to making it the provider's fault. Resolution times can be pretty low when you're playing the undercommunications waiting game.)
I tend to monitor my vendors in great detail, so I'm usually aware of an outage before they are. Nobody that I pay for a plan has any way for me to make my monitoring available to their teams. I had a call with someone once about it (a popular build-your-website-from-a-git-repo service), and they said "oh, we'd let you into our Slack for 5 figures a year". No thanks! Give me a webhook and I can ping you within seconds of your global load balancer going down for the 10th time this year.
Anyway this has become a rant, but fuck undercommunication. It wastes my day, and I am already locked into a contract, so my business isn't going anywhere. Let's be honest now that you have my money!