News for hackers. Stuff that matters.
Re:1.1.1.1 lookup failures on October 4th, 2023 by denysvitali on Wednesday October 04, 2023 @06:36PM (#37772720)
My only concern:

7:57 UTC: first reports coming in

I noticed this issue quite quickly ("reported" at 7:54 UTC [1]), and I noticed I wasn't alone thanks to Twitter / X. I tried to get in touch with Cloudflare to report this issue - but I haven't found any meaningful contact other than Twitter.

For such an important service, I'm impressed there is no contact email / form where you can get in touch with the engineers responsible for keeping the service up and running.

Other than that, kudos for the well written blog post - as always!

[1]: https://nitter.net/DenysVitali/status/1709476961523835246

Re:1.1.1.1 lookup failures on October 4th, 2023 by hackmiester on Wednesday October 04, 2023 @07:05PM (#37772950)
Get what you pay for I guess.
Re:1.1.1.1 lookup failures on October 4th, 2023 by cj on Wednesday October 04, 2023 @09:54PM (#37774078)
This sounds so foreign to my experience.

Even when I just had 2 paid domains with Cloudflare, there was real time chat support that seemed to always have an available engineer when something came up.

Maybe they don't provide support like that to free users.

Re:1.1.1.1 lookup failures on October 4th, 2023 by abraham on Wednesday October 04, 2023 @10:35PM (#37774290)
You have to pay $200/month for access to live chat support.

https://www.cloudflare.com/plans/

Re:1.1.1.1 lookup failures on October 4th, 2023 by denysvitali on Thursday October 05, 2023 @02:43AM (#37775543)
I don't even have a Cloudflare account. You don't need a Cloudflare account to use their public DNS servers
Re:1.1.1.1 lookup failures on October 4th, 2023 by lopkeny12ko on Thursday October 05, 2023 @01:30AM (#37775192)
Do you truly believe that the turnaround time for you contacting support, speaking to a first-tier support agent, having the request escalated to engineering, having a support engineer triaging it to the right team, and finally having the report being gauged so urgent that it pages the oncall for that team, is faster than automated infrastructure monitoring that the teams already have?

For something as core as DNS, I'm sure engineers were aware about the issue within minutes. There's a lot of politics and processes between that and public acknowledgment of an incident.

Re:1.1.1.1 lookup failures on October 4th, 2023 by denysvitali on Thursday October 05, 2023 @02:42AM (#37775534)
The timeline doesn't mention any internal monitor system but "external reports".

What I want as an engineer is an engineering contact (similar to a NOC) that "normal customers" aren't aware of.

Something as critical as DNS should have that - but maybe I expect too much from a free service.

Re:1.1.1.1 lookup failures on October 4th, 2023 by hackmiester on Friday October 06, 2023 @06:05PM (#37796998)
But you are sure of this, even though there were no indications in that direction, and several in the opposite.
Re:1.1.1.1 lookup failures on October 4th, 2023 by sambazi on Thursday October 05, 2023 @02:36AM (#37775506)
> For such an important service, I'm impressed there is no contact email

cloud: you are the product, not the customer

Re:1.1.1.1 lookup failures on October 4th, 2023 by dx034 on Thursday October 05, 2023 @03:10AM (#37775673)
I don't think that statement makes sense here. In what way is the user of 1.1.1.1 the product? And for cloud services in general, they mostly tend to be subscription or usage based. In both cases, the user is clearly the customer.

I get the "you're the product, not the customer" for social networks or ad-based search engines, but certainly not for cloud products.

Re:1.1.1.1 lookup failures on October 4th, 2023 by denysvitali on Thursday October 05, 2023 @04:23AM (#37776115)
And I wouldn't consider 1.1.1.1 to be a "cloud product". You don't need a Cloudflare subscription to use it
Re:1.1.1.1 lookup failures on October 4th, 2023 by sambazi on Sunday October 08, 2023 @02:00PM (#37812853)
dude, there's even cloud in the name of the vendor
Re:1.1.1.1 lookup failures on October 4th, 2023 by laboratorymice on Thursday October 05, 2023 @04:28AM (#37776147)
Might be a stretch for cloud services generally, but for public dns resolvers there's definitely an argument to be made: a) you pay nothing for the service, b) providing the service costs money, c) the provider benefits indirectly from the added insights and market power of user traffic
Re:1.1.1.1 lookup failures on October 4th, 2023 by Wallacy on Thursday October 05, 2023 @10:29AM (#37779060)
Discord is the place to go. All CF Devs are there.

I got on particular fix one time in just a few minutes, and scalated pretty quickly. I got a recomendation fix on my side (a code change on my deployment that cover the problem) and a permanent fix on CF side 4h latter (time that took to changes propagate to all colos)