News for hackers. Stuff that matters.

A comment on RethinkDNS Resolver That Deploys to CF Workers, Deno Deploy, Fastly, Fly.io. Parent comment

Re:RethinkDNS Resolver That Deploys to CF Workers, Deno Depl... by maxloh on Sunday May 04, 2025 @11:50AM (#43887423)
I was able to reproduce your results with the provided command. In that case, Rethink DNS is 1-2 ms faster than Cloudflare DNS. However, the provided command actually runs 300 queries in a really short timeframe (specified by -c 300), which is very different from real-world use cases — in most cases, we look up a domain only once, and the result is cached by the OS.

If we set the count parameter to 1 (closer to the mentioned real-world use cases), we can reproduce the result in my comment above. Rethink DNS actually takes 450 ms to return a response.

  ./godnsbench -c 1 -q google.com -a https://sky.rethinkdns.com/dns-query
  ...
  2025/05/04 23:22:47 [info] Average per query: 452.3948ms
With Cloudflare DNS, it takes only 35 ms, even when using DNS over HTTPS.

  ./godnsbench -c 1 -q google.com -a https://cloudflare-dns.com/dns-query
  ...
  2025/05/04 23:22:49 [info] Average per query: 34.4343ms
As sky.rethinkdns.com is hosted on Cloudflare, there shouldn't be any notable network differences between Rethink DNS and Cloudflare DNS. If you inspect the list of servers in the following two lists, they are almost identical.

- https://check-host.net/check-ping?host=https://sky.rethinkdn...

- https://check-host.net/check-ping?host=https://cloudflare-dn...

If CPU time isn't the bottleneck of response time, any idea what the reason is behind the difference in response time?

[ Parent | on HN ]
Re:RethinkDNS Resolver That Deploys to CF Workers, Deno Depl... by e12e on Sunday May 04, 2025 @02:36PM (#43888505)
Really appreciate both of you commenting here.

Could the ~430ms be startup time for a sleeping worker?

Or is it the cost for a cache miss?

Ed: hm, I suppose google.com should pretty much always be a cache hit... Unless they have some really strange TTLs?

[ Parent | on HN ]
Re:RethinkDNS Resolver That Deploys to CF Workers, Deno Depl... by maxloh on Sunday May 04, 2025 @03:03PM (#43888702)
Actually, 450 ms was already a cache hit. The response time would be 530 ms on the first query of google.com:

  ./godnsbench -c 1 -q google.com -a https://sky.rethinkdns.com/dns-query
  ...
  2025/05/05 02:50:14 [info] Average per query: 532.1144ms
Subsequent queries for google.com within a few minutes would result in a slightly shorter response time, like my second comment above.

  ./godnsbench -c 1 -q google.com -a https://sky.rethinkdns.com/dns-query
  ...
  2025/05/05 02:50:48 [info] Average per query: 468.5749ms

  ./godnsbench -c 1 -q google.com -a https://sky.rethinkdns.com/dns-query
  ...
  2025/05/05 02:55:04 [info] Average per query: 472.1229ms
Some of the lengthy response times originate from the overhead of DNS over HTTPS. If we switch to the DNS over TLS version of Rethink DNS, there will be about a 240-320 ms improvement in response time.

  ./godnsbench -c 1 -q google.com -a tls://max.rethinkdns.com
  ...
  2025/05/05 02:50:26 [info] Average per query: 290.4762ms

  ./godnsbench -c 1 -q google.com -a tls://max.rethinkdns.com
  ...
  2025/05/05 02:50:51 [info] Average per query: 145.1868ms
However, 290ms is still quite a large delay for DNS queries, which makes webpage rendering a quarter of a second slower the first time you visit it.
[ Parent | on HN ]
Re:RethinkDNS Resolver That Deploys to CF Workers, Deno Depl... by e12e on Monday May 05, 2025 @05:14PM (#43899509)
So how is it fast in the multiple lookup case?

Is it all per request overhead and gets eaten by a single large request doing multiple lookups?

[ Parent | on HN ]
Re:RethinkDNS Resolver That Deploys to CF Workers, Deno Depl... by ignoramous on Monday May 05, 2025 @02:12PM (#43897802)
> Could the ~430ms be startup time for a sleeping worker? Or is it the cost for a cache miss?

One won't really gain much insight for DoH / DoT resolver performance with one query per connection, I don't think.

For something as lightweight as domain name resolution, network latency / round-trip time dominate so much (for a public DNS resolver anyway) that the CPU-bound optimizations are almost always pointless if you don't first optimize your network deployment: For example, not running servers closer to your users wouldn't move the needle much regardless of how big your caches are, or the language your resolver is written in (C or Rust or hand-rolled assembly or SIMD/SIMT built-ins etc). A good indicator of performance of a pubic DNS resolver would be to monitor it over a period of time, to effectively rule out noise introduced by variances.

[ Parent | on HN ]
Re:RethinkDNS Resolver That Deploys to CF Workers, Deno Depl... by e12e on Monday May 05, 2025 @05:18PM (#43899553)
But as far as I understand - these tests are run on hosts with comparable distance/ping?

And 400ms ping is remarkably bad for a client that can otherwise do 20ms to a nearby server?

[ Parent | on HN ]
Re:RethinkDNS Resolver That Deploys to CF Workers, Deno Depl... by ignoramous on Tuesday May 06, 2025 @03:49AM (#43902678)
godnsbench isn't testing pings but DoH / DoT (and DoH is expensive due to the TCP, TLS, and HTTP overhead [0]). OP did share a ping report (which paints a different picture): https://check-host.net/check-ping?host=https://sky.rethinkdn...

Sending one DNS query per HTTP connection (which is what the OP claims is "real world") is not how "real world" DoH clients are supposed to be written or work. So it is unlikely to be representative of "real world" delays.

In fact, in my tests, other public DNS resolvers don't fair any better with one query per connection, either.

[0] For example, a TLS handshake transfers 1.5KB to 4KB, while a typical DNS query is 30B to 70B & responses are 70B to 150B.

[ Parent | on HN ]
Re:RethinkDNS Resolver That Deploys to CF Workers, Deno Depl... by ignoramous on Monday May 05, 2025 @01:53PM (#43897644)
> However, the provided command actually runs 300 queries in a really short timeframe (specified by -c 300)

Our servers see 2500-25000 requests per second. So, my intention was to test the server, not the client (original post points out issues with the server).

> which is very different from real-world use cases

Not really. A decent DoH/DoT/ODoH client would pool TLS/HTTP connections (on top of mutliplexing HTTP/2 on TCP) & have its own caches (probably hydrate the top domains in the background, even).

> As sky.rethinkdns.com is hosted on Cloudflare, there shouldn't be any notable network differences between Rethink DNS and Cloudflare DNS.

Cloudflare likely co-locates its DNS resolvers with all major ISPs, which might explain the speed. For instance, ping 1.1.1.1 reports ridiculous numbers (<3ms) on some networks.

> With Cloudflare DNS, it takes only 35 ms, even when using DNS over HTTPS

In my experiments, for (non-random) repeat queries, the fastest Cloudflare response is about 20% faster with "-c 1" than Max and 30% faster than Sky.

You should try if any other non-BigTech public DNS posts comparable numbers for "-c 1": Like ControlD, AdGuard, NextDNS, CleanBrowsing, DNSFilter, Quad9 etc. In my experiments, these aren't significantly better than Max or Sky. And imo, concurrent "random" (cache-busting) queries are better test of a remote resolver's capabilities.

For Max specifically, caches speed up response times significantly, even if Max is deployed to only 30+ locations (as compared to Cloudflare's 300+) running 40+ low-powered servers on Fly.io's network. For Sky, a lot is under the control of Cloudflare: How they route & shape our traffic and just where they run our Workers. For the most part though, Sky is comparatively pretty fast. If you use Rethink (enable Configure -> DNS -> DNS Booster to turn on optimizations), you should be able to see per-query round-trip time in Configure -> Logs -> DNS.

[ Parent | on HN ]