ProtonMail Is Down

Re:ProtonMail Is Down by Jaepa on Monday July 11, 2022 @11:37AM (#32057545)

I'm kind of curious what the source of the outage is.

It looks like most of their services are down, including their incident status page¹. The number of things I can think of that so many different services & are not a massively catastrophic failure is pretty small. Though that may be a lack of imagination on my part.

1: https://status.protontech.ch/incidents/195

[ Parent | on HN ]

Re:ProtonMail Is Down by krn on Monday July 11, 2022 @02:15PM (#32059674)

> The number of things I can think of that so many different services & are not a massively catastrophic failure is pretty small.

Like, for instance, a simple DNS or BGP misconfiguration that takes the entire network down?

Cloudflare (June 2022):

https://news.ycombinator.com/item?id=31823132

OVH (October 2021):

https://news.ycombinator.com/item?id=28849319

Facebook (October 2021):

https://news.ycombinator.com/item?id=28752131

Google (June 2019):

https://news.ycombinator.com/item?id=20077421

[ Parent | on HN ]

Re:ProtonMail Is Down by 29083011397778 on Monday July 11, 2022 @02:28PM (#32059838)

We don't know for certain, but you may have missed Rogers [0] - which brought down cell phones (every function but wifi), debit transactions, and 911 services.

[0] https://en.m.wikipedia.org/wiki/2022_Rogers_Communications_o...

[ Parent | on HN ]

Re:ProtonMail Is Down by tick_tock_tick on Monday July 11, 2022 @03:31PM (#32060666)

No ones it too big to not fuck up BGP or DNS from time to time.

[ Parent | on HN ]

Re:ProtonMail Is Down by willis936 on Monday July 11, 2022 @11:39AM (#32057573)

They have another status page that currently reports everything is up (it isn't).

https://status.proton.me/

[ Parent | on HN ]

Re:ProtonMail Is Down by LeifCarrotson on Monday July 11, 2022 @01:39PM (#32059177)

And another that says that some systems (many systems) are experiencing issues/in a state of "partial outage":

https://protonstatus.com/

[ Parent | on HN ]

Re:ProtonMail Is Down by KineticArms on Monday July 11, 2022 @04:52PM (#32061669)

"An error occurred on client"

[ Parent | on HN ]

Re:ProtonMail Is Down by 0xbadcafebee on Monday July 11, 2022 @02:51PM (#32060140)

  https://protonstatus.com/incidents/195#update-201
  45 minutes ago : Our engineering team is working on a database performance issue which is impacting Proton services

Relational database woes. :( Probably re-indexing or slow queries or something. This is why companies with an RDBMS as a SPOF need a good DBA.

[ Parent | on HN ]

Re:ProtonMail Is Down by 0xbadcafebee on Monday July 11, 2022 @01:25PM (#32059002)

Doesn't have to be catastrophic failure. Most people just assume that such services are architected properly to be HA, but they usually aren't. They also aren't operated as HA usually, so things like a config change are rolled out globally to everything at once, and then everything breaks and they can't roll back, and you have total outage. Very common.

...and of course, they might not be down at all, it might just be (example) a route from the west coast got fucked in a BGP table so the west coast can't access it, and if that's where the majority of their users are that's all the comments you'll see.

[ Parent | on HN ]

Re:ProtonMail Is Down by sllabres on Monday July 11, 2022 @02:18PM (#32059696)

A bit nitpicking but even with a proper implemented HA solution there would be a service interruption.

[ Parent | on HN ]

Re:ProtonMail Is Down by confounded on Monday July 11, 2022 @02:04PM (#32059514)

What is HA?

[ Parent | on HN ]

Re:ProtonMail Is Down by Pryde on Monday July 11, 2022 @02:05PM (#32059531)

High Availability

[ Parent | on HN ]

Re:ProtonMail Is Down by detaro on Monday July 11, 2022 @02:05PM (#32059528)

high availability

[ Parent | on HN ]

Re:ProtonMail Is Down by brewdad on Tuesday July 12, 2022 @01:39AM (#32066076)

Maybe they installed a whole lot of SSDs on the same day 4.5 years ago.

[ Parent | on HN ]

Re:ProtonMail Is Down by IG_Semmelweiss on Monday July 11, 2022 @11:39AM (#32057562)

I'm seeing their status page up now.

But they are down hard otherwise.

[ Parent | on HN ]

Re:ProtonMail Is Down by jjtheblunt on Monday July 11, 2022 @12:28PM (#32058299)

maybe the status page is therefore also down, misleadingly showing its last pre-down state

[ Parent | on HN ]

Re:ProtonMail Is Down by guerrilla on Monday July 11, 2022 @01:22PM (#32058965)

Maybe it's the same hard drive problem a lot of others (including HN) had.

[ Parent | on HN ]