News for hackers. Stuff that matters.
Re:Azure is experiencing DNS issues in all regions by deathanatos on Thursday May 02, 2019 @07:01PM (#19813795)
It's DNS, so it is somewhat inherently global. Route53 isn't region specific either, so I could see an issue with that having a global effect, too.
Re:Azure is experiencing DNS issues in all regions by y0y on Thursday May 02, 2019 @07:18PM (#19813935)
DNS is also inherently distributed. This should make it resilient to all of the most common outage scenarios, and is likely why AWS offers a 100% uptime SLA for Route 53.

I'll be interested in the post-mortem from Azure on this one.

Re:Azure is experiencing DNS issues in all regions by deathanatos on Thursday May 02, 2019 @08:29PM (#19814382)
> likely why AWS offers a 100% uptime SLA for Route 53

Well, that's interesting. We occasionally see getaddrinfo() calls fail claiming domains that we know exist at the failure time (b/c the records are completely static) don't exist. (We've not got a reproducible case for this yet, and it's incredibly rare for any given VM/service. But across our fleet, it crops up fairly regularly.)

Re:Azure is experiencing DNS issues in all regions by donavanm on Friday May 03, 2019 @12:10AM (#19815448)
I used to work on route 53 for a few years. I cant speak to your specific issue. Too much depends on your clients, your networks, your resolvers. But ... turn on query logging at a minimum. You should get a timestamp, qname, and rtype to identify nxdomain.

That said the most common cause of authoritative nxdomain is if youre adding/deleting records and querying them before propagation is complete. You may want to log/poll your rrset change status separately to correlate.

The other is that depending on networks intermediate dns tampering happens all the time. Qname, rname, rtype, all get modified. Responses and queries are duplicated, intercepted, and manipulated. Some good research out of dns oarc and a dude out of australia (iirc).

Re:Azure is experiencing DNS issues in all regions by cthalupa on Thursday May 02, 2019 @10:25PM (#19814969)
> We occasionally see getaddrinfo() calls fail claiming domains that we know exist at the failure time (b/c the records are completely static) don't exist.

That could be whatever resolvers you're hitting failing rather than an issue with Route 53 authoritative nameservers, though. The resolving DNS servers in EC2 are not actually part of Route 53, for example.

Re:Azure is experiencing DNS issues in all regions by deathanatos on Thursday May 02, 2019 @10:59PM (#19815139)
I'd think that would correspond to EAI_AGAIN or EAI_FAIL, whereas I'm pretty sure we're getting a EAI_NONAME.
Re:Azure is experiencing DNS issues in all regions by leesalminen on Thursday May 02, 2019 @10:25PM (#19814973)
We’ve experienced the same thing. I’ve never been able to figure it out. If you ever do, please let me know! I’ll owe you a beer ;)
Re:Azure is experiencing DNS issues in all regions by hfern on Thursday May 02, 2019 @10:29PM (#19814995)
You may be hitting ec2 dns rate limits.
Re:Azure is experiencing DNS issues in all regions by deathanatos on Thursday May 02, 2019 @11:03PM (#19815164)
I would expect EAI_FAIL or EAI_AGAIN, but I'm pretty sure we're getting EAI_NONAME.

But, the stuff that hits this problem the most often is of the quality level that I wouldn't find that terribly surprising. Seems AWS "documents" this as,

> The number of DNS queries per second supported by the Amazon-provided DNS server varies by the type of query, the size of response, and the protocol in use.

How specific.

Re:Azure is experiencing DNS issues in all regions by el_duderino on Thursday May 02, 2019 @07:34PM (#19814055)
Do they typically provide a postmortem?
Re:Azure is experiencing DNS issues in all regions by crankylinuxuser on Thursday May 02, 2019 @10:18PM (#19814931)
It's Microsoft. I'm sure they just rebooted it!

(I had to, see username!)

edit: seriously,-3 ? it was a joke.

Re:Azure is experiencing DNS issues in all regions by aioprisan on Thursday May 02, 2019 @07:17PM (#19813927)
Sure, but that's hypothetical, and I don't recall AWS having any such issue in recent history.