News for hackers. Stuff that matters.
Re:Zoom.us is down by maest on Tuesday June 21, 2022 @10:11AM (#31824163)
https://status.zoom.us is green, even though service is down/severely degraded.
Re:Zoom.us is down by juice_bus on Tuesday June 21, 2022 @10:21AM (#31824247)
What use are status pages when you need to get VP+ clearance to acknowledge an outage? We see this time and time again with major services.

It is a shame that we have to find out about outages on HN.

Re:Zoom.us is down by toomuchtodo on Tuesday June 21, 2022 @10:54AM (#31824636)
> What use are status pages

Marketing and sales. Not /s

We are in dire need of something crowdsourced, or where someone like DataDog or other telemetry systems offer you the ability to share non sensitive metrics publicly for various cloud or SaaS systems that they publish.

Edit: y’all are amazing with these monitoring tools!

Re:Zoom.us is down by CoastalCoder on Tuesday June 21, 2022 @11:00AM (#31824716)
Does downdetector.com meet that need?

At least for Steam, I've found it pretty useful.

Re:Zoom.us is down by toomuchtodo on Tuesday June 21, 2022 @11:07AM (#31824790)
For public facing endpoints or front ends, yes. For complex systems where you’d need sensors inside (AWS, GCP, anything IaaS or PaaS, etc), no.

Somewhat but not entirely similar to BGP looking glass systems.

Re:Zoom.us is down by geocrasher on Tuesday June 21, 2022 @11:29AM (#31825056)
You mean like https://mcbroken.com/ ?
Re:Zoom.us is down by WarOnPrivacy on Tuesday June 21, 2022 @12:40PM (#31825995)
Oooooo. OpenStreetMap has clearly visible county lines. I'm in lust.
Re:Zoom.us is down by i67vw3 on Tuesday June 21, 2022 @12:20PM (#31825710)
Downdetector was itself down today for half an hour when the cloudflare incident happened.
Re:Zoom.us is down by lamontcg on Tuesday June 21, 2022 @01:23PM (#31826612)
> Marketing and sales. Not /s

When I got hired at Amazon in 2001 we had a "gonefishin" page that was a static page that would be served in the event of an outage (this was before status pages, but it was kind of the same thing -- public acknowledgement of a major incident). The standard protocol was within minutes of a sev 1 to make a decision to display the GF page once it was confirmed that the whole site was down and then work to fix the issue.

By the time I left in 2006 that was no longer policy since reporters had setup monitoring for that page to detect outages and report on service availability so they just let it crash and return 500s or whatever the failure mode was. Optimize for making the job of external agencies doing reporting on their availability harder instead of easier.

Re:Zoom.us is down by mbesto on Tuesday June 21, 2022 @11:44AM (#31825255)
> We are in dire need of something crowdsourced, or where someone like DataDog or other telemetry systems offer you the ability to share non sensitive metrics publicly for various cloud or SaaS systems that they publish.

This is literally what I'm building right now. See reply above: https://news.ycombinator.com/item?id=31825239

Shoot me an email if anyone is interested in getting beta access.

Re:Zoom.us is down by austinpena on Tuesday June 21, 2022 @12:14PM (#31825641)
Built something like that at https://taloflow.ai/is-aws-down

Looks like there’s been some errors too

Re:Zoom.us is down by cmccart on Tuesday June 21, 2022 @01:01PM (#31826324)
New Relic has had this for a few years now for a couple hundred of the most requested domains: https://docs.newrelic.com/docs/query-your-data/explore-query...

Disclaimer - I work at New Relic but not on this.

Re:Zoom.us is down by jmartens on Tuesday June 21, 2022 @12:40PM (#31826002)
That's exactly what we are building at https://metrist.io/
Re:Zoom.us is down by jabroni_salad on Tuesday June 21, 2022 @01:12PM (#31826474)
I use uptimerobot to monitor a lot of endpoints that I depend upon but don't really control. Been burned by first party status pages way too many times.
Re:Zoom.us is down by philote on Tuesday June 21, 2022 @11:08AM (#31824808)
I guess one could argue it isn't an outage since it only seems to have affected a subset of users. I got on a zoom call when this issue started and we had 3 of the 4 participants. Only one couldn't connect due to the issue.

But I do agree they should be able to monitor things better and show some sort of update on their status page as soon as possible.

Re:Zoom.us is down by Wowfunhappy on Tuesday June 21, 2022 @11:15AM (#31824899)
I feel like this is almost worse. It would be awful if you were the only person who couldn't connect to a high-stakes meeting. At least if it happens to everyone, it's obvious that the problem is on Zoom's end.
Re:Zoom.us is down by CaptainZapp on Tuesday June 21, 2022 @01:02PM (#31826335)
No problem for us.

We use Skype for Business, which is so flaky at times that the default assumption if somebody is not joining is that the system conked on her.

Re:Zoom.us is down by danachow on Tuesday June 21, 2022 @12:18PM (#31825695)
If you stake your life’s happiness on pleasing morons (and the morons in this case are those that pretty much don’t immediately assume technical problems out of your control) - you’re pretty much guaranteed a bad time.
Re:Zoom.us is down by Wowfunhappy on Tuesday June 21, 2022 @12:42PM (#31826032)
A couple of months ago, I finally landed a first-round job interview at a place where I've wanted to work for several years. The interview was conducted over Zoom.

What would have happened if Zoom had worked fine on their end, but I was randomly unable to connect? Perhaps it would have been fine—they would have been understanding, and we would have rescheduled for another day. Perhaps if they hadn't been understanding, I shouldn't have wanted to work for them anyway.

But, I don't know. I wanted to work for them, and I was competing with other candidates who presumably interviewed on different days. Hiring processes are inherently imperfect, and lots of things can be consciously or unconsciously treated as a red flag.

(And yes, lots of other things could have happened on the day of the interview. But I still find this scenario particularly scary to think about.)

Re:Zoom.us is down by danachow on Tuesday June 21, 2022 @12:56PM (#31826235)
> Hiring processes are inherently imperfect, and lots of things can be consciously or unconsciously treated as a red flag.

Exactly, so it’s weird to worry about a Zoom problem in particular. If anything it’s a little better now since most people are conditioned to think of technical problems as less likely the affected persons fault (that’s why I referred to the alternative as “morons”) - even if you left yourself plenty of time and did everything right and public transit fucked you over it was never a good look.

Re:Zoom.us is down by mkl95 on Tuesday June 21, 2022 @12:17PM (#31825681)
Whether you consider it an outage or not seems to be a political / PR thing these days. I used to work on a SaaS that relied on a handful of big customers to make payroll. If their favourite stuff stopped working, hell ensued. On the other hand Atlassian pretended nothing was going on for a while recently, because they could afford to lose 400 customers.
Re:Zoom.us is down by reaperducer on Tuesday June 21, 2022 @01:03PM (#31826345)
I guess one could argue it isn't an outage since it only seems to have affected a subset of users

If an electric company serving a million people leaves 100 of them in the dark, it's still an "outage."

Why give a free pass to Zoom? Because it's a tech company, and we've been trained to accept failures as the cost of admission?

Re:Zoom.us is down by interestica on Tuesday June 21, 2022 @11:22AM (#31824977)
So for you it was only a 25% outage?
Re:Zoom.us is down by no_wizard on Tuesday June 21, 2022 @12:25PM (#31825773)
This has made me realize why companies like pingdom have a business. I've always wondered, in the sense that I couldn't quite understand why you'd pay for someone just to ping things and alert you of outages (this was early in my career)

But over the last 4 years specifically I not only understand it I can't imagine not having a service like it.

Disclaimer: I don't work for pingdom and my current company doesn't use their services, I have in the past, they're pretty good, but I'm just using them as an example here

Re:Zoom.us is down by jmartens on Tuesday June 21, 2022 @12:33PM (#31825889)
If you like pingdom, you'll love what we are building at Metrist https://metrist.io/
Re:Zoom.us is down by mbesto on Tuesday June 21, 2022 @11:43AM (#31825239)
I'm actually building a product to solve this. If anyone is interested in beta testing, we should be rolling this out in 2~3 weeks. Shoot me an email: mbesto @ gmail service
Re:Zoom.us is down by gjsman-1000 on Tuesday June 21, 2022 @10:59AM (#31824698)
Is it so hard? At my company, I set up a status page linked to a pinging service which automatically pings various endpoints every 5 minutes (as well as our 3rd-party dependencies) and automatically flags any problems if a ping does not respond.

The pinging service and status page, at our scale, is free and our status page is actually useful and automated for 90% of our stuff.

Re:Zoom.us is down by cortesoft on Tuesday June 21, 2022 @11:04AM (#31824761)
That works for a website, but doesn’t work for all outage types with a service like zoom. If the website/api is responding, but you can’t create new calls, for example, the outage wouldn’t be detected.
Re:Zoom.us is down by phaer on Tuesday June 21, 2022 @11:08AM (#31824810)
I think the problem might be less on the technical and more on the business-side of things.

Status pages that raise customers confidence in your service are good from a marketing perspective.

Automatically publishing uptime data without human review might be bad from a marketing perspective, if you don't trust the engineering department to actually deliver or if your service depends on too many external dependencies.

Re:Zoom.us is down by treeman79 on Tuesday June 21, 2022 @11:07AM (#31824803)
I’ve come across a lot of websites that respond to health checks just fine. Because the page health check doesn’t hit the database or other services.
Re:Zoom.us is down by digitallyfree on Tuesday June 21, 2022 @11:24AM (#31825009)
I do this at home using LMNS, very useful to detect service failures as well as latency spikes. I ping my upstream ISP router, Google, Cloudflare, my DNS provider, and several others.

Note that as another commenter said this only tells if the server is up, not if the service is working properly or not. In this case it won't work since the Zoom website is loading fine but the meetings don't work properly.

Re:Zoom.us is down by mason55 on Tuesday June 21, 2022 @11:06AM (#31824777)
The update lag has nothing to do with technical abilities. It's about things like marketing and contractual SLAs.

If acknowledging downtime causes you to violate an SLA and pay a bunch of penalties then you don't want some automated script to trigger it.

Re:Zoom.us is down by gjsman-1000 on Tuesday June 21, 2022 @11:08AM (#31824814)
> violate an SLA and pay a bunch of penalties

The SLA would not depend on what the status page reported, but the actual downtime. If the script malfunctioned, you wouldn't need to pay out because it wasn't actual downtime contractually - and if it was actual downtime, I guess it makes it harder to squeeze around but that's only if someone is carefully taking legally admissible evidence from the first minute the status page reports (which screenshots alone don't always meet).

Re:Zoom.us is down by kull on Tuesday June 21, 2022 @10:20AM (#31824240)
Service status pages are so useless.
Re:Zoom.us is down by lxchase on Tuesday June 21, 2022 @10:29AM (#31824347)
At this point, I just default to downdetector
Re:Zoom.us is down by WarOnPrivacy on Tuesday June 21, 2022 @12:46PM (#31826091)
It's for nostalgia - the one bit of early web that's still unchanged.