Not sure about the validity these arguments. Yes, Google is likely to have more cached data than you, but they can also be a half a country or two away.
In my experience caches matter less than one might think, since more popular data is usually the one that's low latency anyway.
> Privacy? With DNS over TLS/DNS over HTTPS, your ISPs can't see
Let's agree on one thing: That surf data is a lot more valuable to Google than most other actors, including your ISP, because they're the ones in a position to monetize it.
Your ISP has more data than they know what to do with anyway. Should they try to monetize it despite the murky legal waters (they really wouldn't want to knowingly help copyright infringement, for example), the realistic option would be for them to sell it to someone very much like Google. It should not come as a surprise that the latter is happy to shortcut the process.
Google has edge nodes in pretty much every ISP of every country except China. A query to Google's public DNS never leaves your ISP, much less your country.
>When clients send queries to Google Public DNS, they are routed to the nearest location advertising the anycast address used (8.8.8.8, 8.8.4.4, or one of the IPv6 addresses in 2001:4860:4860::). The specific locations advertising these anycast addresses change due to network conditions and traffic load, and include nearly all of the Core data centers and Edge Points of Presence (PoPs) in the Google Edge Network.
For hits that are in the cache (the usual case) it's obviously faster than going out to the 'net,. The black-holing combined with ad-blockers mean browsing is a lot faster and considerably more peaceful.
In terms of maintenance, it's no real effort other than updating every few years (being behind NAT the security risks aren't huge) and it means the entire household sees the benefit (plus access to webservers/wiki's etc on the .local domain).
Obviously the downside is it needs to be on something that runs 24x7 and a modicum of IT skills are required to set it up. One other catch is that your ISP might block you for not using their DNS; BT (UK ISP) did this, but it is possible to turn off this 'security feature' via a rather obfuscated web page (may have changed since I last did it).
Erm ... what? How does that work? If they don't see DNS requests from you at their resolver for a week, they disable your connection?!
I assume they detected it simply by seeing DNS queries going to non-BT servers. Note this was a few years ago when it was pretty common for PC malware to hijack DNS requests, so could be it's changed in the meantime.
N.B. I also recall BT redirecting requests for non-existent domains to some partner of theirs, I assume experimentally as I haven't seen or heard of that for a while.
They are still doing this. I ran into it just yesterday. They do however make it very easy (click a couple of links) to turn it off.
Thanks for the update (if somewhat depressing to know they still get away with it).
Mine forces me to use their DNS, would love to turn it off at a router level. I know I can buy a new router but I can't justify that right now.
It's perfectly possible to recursive resolve your misses to Google's DNS server if you want.
I don't think that's how recursive name servers work. It's been a while since I've had to reason about this but, for example if 'www.google.com' in not cache, it contacts a root domain server, then a 'com' domain server, then a 'google.com' domain server, which finally answers the query, which then gets cached by the recursive name server.
Eventually the recursive name server on your family or organization's local LAN will have a decent cache hit ratio, and the round trip times to your local recursive server could be potentially an order of magnitude (2ms vs. 20ms) faster than talking to Google or CloudFlare.
It's possible that your ISP can still know which DNS lookups you're doing by snooping the traffic between your recursive DNS server and other DNS servers on the net, but I'm guessing that they're not doing this because it's not as easy as just ingesting their own DNS logs.
Almost all DNS servers ever have an ability to set a DNS forwarder instead of using root hints.
Unfortunately, that hasn’t been open source’s forte historically.
I suppose we could come up with some easy configurable templates that would automatically install servers for privacy conscious individuals, but if at some point something goes wrong, most people are going to be stuck without service and no easy fix.
In the case of DNS, it’s dead easy if the server is built in to home routers.
Many of them do, in fact, ship with local DNS already - it’s just that many users override it with 8.8.8.8 or whatever.
However, even if we did switch everyone to running their own DNS resolvers, what would happen then? Without the massive shared caching we have today, the load would significantly increase on the authorative DNS servers for each domain. Even with client local caching.
So the number of companies running their own authorative DNS servers would probably decrease — more of them would be using hosted DNS provided by a third party. A lot of companies host the authorative DNS of their domains with a third party big already. Including myself — I use Cloudflare for all my sites because of the HTTP caching and other things they offer on top of hosted DNS.
Increased load on authorative servers will likely lead to further centralization of DNS hosting with a few big providers IMO. Because even a lot companies that specialize in hosting DNS might not be able to handle the load when everyone is running their own resolver. Only the big DNS hosting companies will be able to afford it. So we end up with everyone hosting DNS with a few DNS hosting providers — Cloudflare, Amazon Route 53, etc.
So by decentralizing the DNS resolvers that clients use, you push companies to centralize the authorative DNS servers further. The net effect is that you will only have shifted where in the resolution the queries centralize.
And let’s say that this happens and Google sees the amount of queries received by 8.8.8.8 drop to near zero over night. Odds are that if Google values the data they gain from clients using these resolvers, they will make a big push to ensure that they host the DNS for as many companies as possible, so that they still end up with their hands on the query data. (And Google does value this data — otherwise they wouldn’t still be offering public DNS query servers.)
And also, what about the root servers? Will they be able to handle the massive increase in load? And won’t the root server traffic be subject to surveillance by state actors wanting to know what sites someone is browsing?
DNS is kind of funny because in a way it is both centralized and decentralized at the same time. But if you want the web to be truly decentralized I believe for the reasons stated above that having people run their own DNS resolvers is not part of the solution.
You are going to have to replace DNS altogether. Realistically I don’t think DNS is going away anytime soon. The web and the internet in general is too reliant on it. But I really wish we could.
"your ISPs can't see what you are doing". If they're analyzing traffic, they can, and if they're doing that, they can see to whom I'm connecting, anyway. But you say nothing about why we should trust Google or Cloudflare. I trust my ISP to be big and dumb. I trust Goole and Cloudflare to want to make money.
FYI I run my own DNS server anyway.
Take news.ycombinator.com, for example: The A record has a TTL of 300 seconds. So, after the first visit, which probably will take a bit longer than asking Google/CF, for every request in the next five minutes, you will have a reduced lookup latency.
Then, after five minutes, the next lookup will go out to the authoritative server. But mind you that the NS records for ycombinator.com have a TTL of two days, so those are still cached, and the refresh is indeed a single request to the authoritative server--which more often than not takes about as long as a cache hit from the recursive Google/CF resolvers (it's also one round trip to the internet ...).
And then, there is stuff like BIND's prefetch mechanism which will start the refresh of an expiring record when it sees a query for that record shortly before its expiry: That query is answered immediately from the cache, and a refresh is started in the background, so that the refreshed record should arrive in the cache before the old version expires ... thus completely eliminating the lookup latency for often-used records. Though you might need to tune it to trigger earlier for your personal use than in the default configuration ...
Yes, a lot of DNS does live in their cache, quite literally.
As the powers that be continue to centralize and exert control in a negative way, I have a feeling the pendulum will swing the other way once people get annoyed with it.
Kids are already using VPNs to circumvent controls.
Need to make a cyberpunk-esque decentralization kit for the next generation to adopt.
I feel like one of the surprisingly-big barriers is just the difficulty of geting a static IP address assigned to your home. If you could do that, then (I think) they could run everything from a Raspberry Pi: their own website, hosting their own email, etc etc.
(It's actually not much harder technically to set this all up on a VPS, but then the kid has to put a monthly fee on a card -- probably a big barrier for parents.)
Am I right, or is there an easy way to get around the static IP issue?
Maybe a Distributed Hash Table DNS interface is in order? I think it could work if you cache your peers, and reach out to let them know your current state. Even during the Venezuela blackouts, not all IPs went dark.
I don't have a clue what a "Distributed Hash Table DNS interface" is, but I'll note that a solution [0] to this problem has been around for over two decades.
I was thinking it would be interesting to have a protocol (probably built on bittorrent) that encrypts your data and backs it up onto other's servers in return for storing some of their encrypted backups on yours.
But many people rely on spamlists, ie lists of ips known to relay spam. The proble with this is twofold:
1. Some people just took the authority to decide who sends spam and who does not. If you get on one of those lists, usually you have to get in touch and pay to get out.
2. Such people usually include residential ip subnets by default, for no technical reason whatsoever.
So in the end my mailserver at home has been in a spamlist for years, even though i never relayed spam and was very careful to configure outbound relay authentication/authorisation, spf, dkim etc.
Are you talking about residential or about dynamic? Because there kinda is a reason for this for dynamic addresses (PC malware sending spam, and the impossibility to list the particular affected PC because it's constantly changing addreses, so you only can block all the addresses those PCs could be using).
If you do have static addresses, whether residential or not, those should not be listed in dialup block lists.
If your home internet connection has dynamic addresses, you still can run your mail server at home by renting some tiny VPS and tunneling its addresses to your home server ...
Practical DIY SMTP is a lost battle. Unless you're in it for the experience of the hosting itself.
Any one of the thousands of VPS hosters that are not one of the half dozen huge "cloud server" companies?
> Practical DIY SMTP is a lost battle. Unless you're in it for the experience of the hosting itself.
No, it's very much not, it works perfectly fine. Or at least well enough--arguably, you should be able to send directly from dialup hosts with a well-established domain and SPF, so things aren't as good as they could be, but far from what some people claim.
Indeed my residential internet connection has a static address but it still gets flagged for spam because it’s inside a residential subnet.
As in? I mean, what makes it a "residential subnet"? Have you tried talking to your ISP about this?
Some resolvers (like unbound) can be configured to prefetch cached entries before they expire. And in any case if you use something locally you are probably going to configure it to fetch recursively from one of the "big guys"
Is this true? They still can see what IPs you're connecting to can't they?
Use DNS Crypt Proxy. It acts as a DNS forwarder, cache and ad blocker for your entire network whilst also encrypting your lookups to 8.8.8.8 and 1.1 (Fun fact: did you know 1.1 is short for CloudFlare 1.0.0.1 BTW?)
The server kept running out of memory and shutting down MySQL so my sites stopped working. I started to learn how to read logs and saw that there is a huge amount of malicious activity directed at my server all the time.
Yesterday I installed Fail2ban which meant installing postfix. I got it working (took most of the day) but I can't send emails to my gmail account because I need to set up dkim and dmarc and other stuff. I have a list. But first I have to learn how to do all this stuff.
I use Fedora every day at work but I'm obviously no sysadmin and you are so right. All this takes hours and hours to learn and then to stay on top of it. When I was on shared hosting I had a lot less control and options but it was a lot easier as well.
Why do you need dkim or dmarc to send to gmail? If you send a test php mail does gmail pick it up?
gmail-smtp-in.l.google.com[173.194.207.27] said: 550-5.7.1 This message does not have authentication information or fails to pass 550-5.7.1 authentication checks. To best protect our users from spam, the 550-5.7.1 message has been blocked. Please visit 550-5.7.1 https://support.google.com/mail/answer/81126#authentication for more 550 5.7.1 information. d203si1756652qkb.228 - gsmtp (in reply to end of DATA command)
there are tons of guides on troubleshooting mysql resource issues. Are they great? I don't know. Have I tried what is mentioned in many of them? Yes. I still have issues. I don't think it's just mysql though. I think it is a lot of little things that I'm slowly eliminating one by one.
http://digitaloceanvps.blogspot.com/2014/04/best-configurati...
For gmail interesting.. wonder why google trusts me.
Easy to test though. Run dnsmasq and enable query logging and see how often it’s having to forward requests. Then realize a recursing resolver is potentially having to go all the way to the TLDs for those requests.
Unless you use a VPN, but that is a different story.
But I don't think we are ever going back to that direction. The arguments and benefits for running one locally are not enough the trouble as well.
Performance? Due to DNS caching at the resolver level, it is probably faster to use Google's 8.8.8.8 or CloudFlare's 1.1.1.1, than anything local (where all dns requests are a MISS).
Privacy? With DNS over TLS/DNS over HTTPS, your ISPs can't see what you are doing. If you run DNS locally, they can. Yes, they will see all the requests your resolvers are doing to the auth DNS servers.
Security? Some good resolvers, like Quad9 or CleanBrowsing will block malicious domains. CleanBrowsing will also help blocking adult content if you have kids. I don't think maintaining such control is practical for most people (pi-hole helps, but still hard to keep it updated and find good enough databases to use).
I would love a de-centralized web, but it is pretty hard to go back.