News for hackers. Stuff that matters.
Re:Cloudflare outage on July 17, 2020 by deathanatos on Saturday July 18, 2020 @12:48AM (#23879035)
> and use short-TTL weighted CNAMEs, defaulting to say, 99% Cloudflare, 1% your internal load balancer. The minute Cloudflare seems problematic, make it 0% 100% to bypass Cloudflare’s infrastructure completely.

Except if you're using CF for DNS service, this wouldn't have worked, as both CF's website & DNS servers were impacted by the outage.

Re:Cloudflare outage on July 17, 2020 by txcwpalpha on Saturday July 18, 2020 @01:05AM (#23879119)
That can't be possible, CF's website explicitly says that their DNS is "always available" with "unparalleled redundancy and 100% uptime"! ;)

In all seriousness, I wonder if they are going to have to change the marketing on the site now...

https://www.cloudflare.com/dns/

Re:Cloudflare outage on July 17, 2020 by hunter2_ on Saturday July 18, 2020 @01:31AM (#23879254)
Somewhere, someone is saying the marketing is fine since 100 has just 1 significant figure and therefore two nines -- no, one and a half nines -- can safely be written as 100.
Re:Cloudflare outage on July 17, 2020 by esperent on Saturday July 18, 2020 @02:30AM (#23879506)
If they are truthful it will now probably change to 99.997% uptime or similar. I expect that's still good compared to many DNS providers.
Re:Cloudflare outage on July 17, 2020 by belorn on Saturday July 18, 2020 @09:39AM (#23881326)
Some dns provider only have one name server which is naturally bad.

However it is a bit bad of CF that a single configuration error can bring all the slave servers down. It mean that they have no redundancy in term of BGP misstakes. Customers of CF that want to avoid this would benefit to add an additional slave server outside the hand of CF.

Zonemaster (dns sanity checking tool) actually complain about CF hosted domain names because of the lack of AS redundancy. The outage yesterday demonstrate nicely why that is an concern and why one should care. https://zonemaster.iis.se/?resultid=7d1fab165987e195

Re:Cloudflare outage on July 17, 2020 by jontro on Saturday July 18, 2020 @10:30AM (#23881600)
Same goes for route53 too unfortunately
Re:Cloudflare outage on July 17, 2020 by QuentinM on Saturday July 18, 2020 @01:01AM (#23879098)
Yeap, that's specifically what I also implicitly meant by "Spread your name servers" (besides having them distributed). To use this technique, you also must have a "Business" account with Cloudflare ($200/mo), so to leverage their ability to front your websites using CNAMEs : )
Re:Cloudflare outage on July 17, 2020 by deathanatos on Saturday July 18, 2020 @01:35AM (#23879275)
Ah, I see.

This got me Googling, and best as I can tell, CF doesn't support zone transfers. (They support being a client, but not a server. So, they could function as one's secondary system, but not as the primary.)

Re:Cloudflare outage on July 17, 2020 by stedaniels on Saturday July 18, 2020 @05:17AM (#23880142)
I imagine/hope a lot of the heavily engineered ops teams don’t use zone transfers, instead using APIs. The idea of primary and secondary wouldn’t really a strong indicator at that scale.
Re:Cloudflare outage on July 17, 2020 by deathanatos on Saturday July 18, 2020 @01:47PM (#23883172)
Sure, I could use the CF APIs… but to do what? AFAICT with a quick look over the documentation, there isn't any way to tail a log of changes being made to the zone. (You can export the entire thing in BIND format, though, the example in the docs has several errors in it that make me wonder how well that would work.) (The idea with zone xfers is that it is at least semi-standardized, whereas CF's API, while useful, is not.)

Then I'm stuck with a bunch of bad questions about how often to poll, and whether CF's rate limits would support any reasonably quick poll interval.

(The big problem is that we have other tooling that relies on being able to update DNS, the big one being ACME for certificate renewal. The changes it makes to CF would need to be rapidly replicated out to the nameserver.)

(Nothing in the post really strikes me as particular to CF, either. I think I could easily replace everything I've said here with "Route 53" and end up in the same bucket, maybe plus or minus zone xfers working.)

Re:Cloudflare outage on July 17, 2020 by foobarbazetc on Saturday July 18, 2020 @01:35PM (#23883063)
Using CF for DNS is, IMHO, a bad idea in general, especially for large sites.

We use AWS + Azure + GCP (yes, all 3) as our authoritative NS and keep them all in sync with octodns.

Re:Cloudflare outage on July 17, 2020 by neillyons on Saturday July 18, 2020 @03:52AM (#23879813)
Ah ha, This probably explains why my internet stopped working for a bit. I had the 1.1.1.1 app from Cloudflare installed on my phone.