Browser-based tracking, on the other hand, can see just about everything, because it's looking at the state of the data after it's been decrypted. And it can, with a reasonable degree of confidence, individually identify people, even when more than one person shares an internet connection, and even when one person uses more than one device or connects to the Internet from more than one location. The higher fidelity of that signal does imply that it's a greater privacy threat.
Also many ISPs are also carriers, which makes things worse.
Source: worked for telecos, have seen a lot of shady stuff myself.
A more far-fetched attack is a sort of timing attack: if you first visit arstechnica.com and then shortly afterwards visit Amazon.com, one could look for links to Amazon on arstechnica and from there have a decent guess what product you viewed on Amazon. This becomes a lot more feasible when paired with the first attack mentioned above.
Hoarding itself is beyond the sophistication of most service providers in the present day.
https://www.telegraph.co.uk/technology/news/8438461/BT-and-P...
Now they might think differently because there is a market for info. Thank you government...
If anyone knows any good resources to learn about the ISP nuts and bolts that make internet magic happen between my modem and everyone else’s servers I would be most appreciative.
As far as Comcast, I'm stuck with them, too. At least in my experience, they don't monkey with DNS - I run and use my own DNS servers, and have never seen interference.
They do run deep packet inspection, and if they detect you, for instance, torrenting commercial media, they'll inject scary messages in port 80 traffic. Given that nearly all web traffic is encrypted now, the main effect of this is to break things like automated `apt-get update`s.
One thing you can do to detect transparent DNS hijacking is to ask a nonexistent server a question. Something like `dig @13.14.15.16 news.ycombinator.com` should not give you an answer. If it does, someone's spying on and/or gaslighting you.
Fairly googleable with "Comcast sandvine". Afaik they haven't done anything like that for years, though.
Bittorrent protocol encryption is only useful to protest against the use of DPI for bandwidth shaping, it has no influence on privacy.
Even with (the weak) encryption, connections to trackers and DHT nodes are easily identified
Well that's vague. What are the symptoms? How would comcast even know that you changed DNS settings? It's possible to infer that from DNS queries to their servers dropping off and traffic to 1.1.1.1 or 8.8.4.4 increasing, but I doubt comcast is competent enough to build that sort of detection system.
Specifically for detecting if a user is not using their DNS, yes you could correlate a user's http requests (unless you are using ESNI the requested domain is in plaintext by design) with traffic logs on their DNS server and observe that there was no DNS request to the ISP DNS server before a request was made, I don't think that would be necessary. Most users use the ISP default DNS - that's your baseline. If most customers hit your DNS X times per Mb of web traffic, then someone using a custom DNS is going to stand out like a sore thumb.
Again, 100% agree that ISPs are not very technically competent (to put it mildly), but as time marches on the ability to both capture and more importantly analyze and report on that data is becoming cheaper and easier. ISPs want to get value from (sell) your data and vendors want to sell ISPs subscriptions to analytics and other platforms that bring them reoccurring revenue. Data from customer DNS is one of the most valuable sources of information an ISP has and I would be surprised if there was not at least an attempt to know how many customers did not use it.
Looking at you, Chromecast that tries 8.8.8.8 40 times an hour even though you know perfectly damn well that 10.10.10.1 is working
And with newer decides that use DoH, you can no longer prevent devices from contacting their own DNS provider without totally firewalling them (or perhaps using some IP blacklist or whitelist, if available?)
Everything that I don't have complete visibility into the network stack of goes on a VLAN that does not forward traffic to the internet, it advertises a proxy via WPAD and DHCP option 252. I have a whitelist of hostnames that each device is allowed to make CONNECT requests to, so far there is only one.
If it's not a plain unencrypted HTTP request to my proxy, or a CONNECT request involving a server/device pair I've decided to trust, it's not going anywhere.
This breaks a lot of things that I would just as soon rather do without. I can't change my universal remote hub settings from the vendor portal, boo-hoo. I can't view my cameras from the hardened VLAN or from the internet (unless I VPN in first since the only copy of the recordings is on my local NAS)... good.
I have an "XFi Gateway" combination modem/router provided by Comcast (perhaps my first mistake) so the DNS settings are restricted and cannot be changed. I have the Comcast modem/router set to bridge mode and connected my own router where I can control the DNS settings.
My understanding is the DNS settings closer to the client control. So in addition to having set my router to Cloudflare's DNS I also set my devices as well. One day, maybe a year ago or so, I'm on HN and I click an archive.is link, read the article, and go to the discussion thread only to see several comments about how archive.is is blocked by Cloudflare DNS. I checked the DNS settings on my MacBook and router and I was indeed using Cloudflare DNS but for some reason I was able to access the "blocked" address.
So I went to the terminal, cleared the cache, and checked nslookup archive.is and it responded correctly. Then I checked a nonsense DNS server: nslookup archive.is 5.9.3.7 or something and it still responded correctly. I tried the same with different websites and got the same result. So I searched "see my DNS server" or something and found a few websites but they all showed Cloudflare. Very odd.
When I logged in with my VPN, Mullvad, and changed the DNS settings on the router and my laptop to Mullvad's and repeated the experiment it finally returned NXDOMAIN. Then I disconnected from the VPN but left Mullvad's DNS settings, repeated the experiment again with the same results - even when I was using a totally bogus DNS server it was returning the correct IP address.
That's when I installed Wireshark and, lo and behold, I could see the requests that should have been going to 1.1.1.1 or 5.9.3.7 going to 75.75.75.75. Comcast.
A call to Comcast was, as expected, a complete waste of time. First they told me it was using their DNS settings because of "their firewall" and then they told me that if I used their built-in router rather than mine + bridge mode I wouldn't have the issue at all.
Messing around in Wireshark I eventually determined the issue had something to do with one specific port that was making the requests (I can't recall how but I think because I could see Mullvad VPN was using a different port for DNS?) so I fiddled around and forced (or maybe redirected?) my router to use that port too and that finally worked in avoiding the Comcast servers. But, knowing just enough to be dangerous and not entirely sure what I was doing, I didn't keep the forced port and decided I'd have to get my own modem and use my VPN in the meantime.
Before I had gotten around to buying a new modem (this was somewhat early in the pandemic) I saw a post on HN about NextDNS and decided I'd see if I ran into the same issue. I didn't, as far as I could tell at least. When I run Wireshark now (I still use NextDNS) I don't see any contact with 75.75.75.75 or 75.75.76.76. I think this is because NextDNS uses DoH? But who knows.
Like I said, I only know enough to be dangerous so perhaps I just had something configured in an odd way that made the Comcast servers step in as a fail-safe and there's a totally innocent explanation. But based on my experience as a Comcast customer I don't really think they're deserving of the benefit of the doubt so I've definitely got a bit of a tin foil hat when it comes to them secretly messing around with my traffic through the leased modem.
https://datatracker.ietf.org/doc/html/draft-livingood-dns-re...
Their own words
It's less common, I think, because more people know how to check their speed than change their DNS.
I know the best answer for a Comcast DNS server in New York is the server I physically installed in a New York Comcast rack, but when a public DNS server asks me from Paris, maybe I suggest a London server, 'cos that's pretty close to Paris, shame that New York isn't.
EDNS Client Subnet is a feature that lets a DNS server say OK, I'm asking on behalf of somebody from 10.20.30/24 and so my system can do the same trick with ECS. But doing this unwinds most of the privacy benefit of using a public service, so several famous public DNS servers explicitly do not use ECS.
Obviously the cheap bulk host used for some Single Serving site like "Is pizza rat mayor of New York yet?" isn't affected, that is only one server and it is wherever it is, but somebody like Netflix absolutely is affected by this because they have their machines close to the customers to deliver better performance and if they don't know where the customer is that inteferes.
QUIC has an optional feature called Connection Migration to help improve this, the remote server is like "Um, now that you're connected to www.example.com here in Glasgow, Scotland, I notice your IP address is from Tokyo, Japan, and this is just a suggestion, but maybe talk to my identical twin also named www.example.com in Tokyo, Japan for better performance? Here is the IP address to try"
Or, somehow more cynically, the ISP makes money from selling the data collected from DNS, so punishes people who use a different DNS provider. (DNS is plaintext-by-default, so I don't quite see how this would work, but it's possible.)
Or perhaps the system uses DNS lookups as a proxy for “is a human browsing the web”; if there aren't enough, it's clearly some kind of automated computer program that doesn't deserve internet access.
https://arstechnica.com/tech-policy/2009/08/comcasts-dns-red...
That's aside from all the political shenanigans they pull, trying to kill municipal broadband and net neutrality. There's scarcely a more evil ISP.
I switched away from Comcast as quickly as I could, and never had trouble with either smaller cable companies or fiber providers.
EDIT: I wasn't thinking, OP is completely right. Sorry for the snark.
This is very outdated, at least for a significant number of smartphones (including all iPhones, but not limited just to those). Apple and IIRC other manufacturers long since isolated the baseband, treating it simply as a standard USB or PCIe peripheral (and in the latter case using an IOMMU with it amongst other things). It has zero special access to anything on the rest of the phone which in the smart phone era is where everything of interest actually lives and happens.
Source - used to certify these things in a lab environment.
It's profitable enough that there seem to be ads for it baked into everything. I won't repeat their name here, but have you avoided that "Sponsored by NxxxVPN" all over the Internet/baked into every YouTube video that has sponsored videos?
There is also hardly anything you can do about from your side. Using a vpn or similar solutions is only shifting the problem from one provider to another. You can reduce the exposure with some measurments, but they are also expensive and complicated.
But for this (and other) reasons companies have started to fix it from the server-side by offering encrypted connections and working on ways to hide your trail from the middleman and their attatched agencies.
An ISP might not know what a user does at pornhub.com, but the ISP does know when and how often the user visits pornhub.com and how much data is exchanged when they do. I'm sure someone would pay for that kind of fingerprinting.
IMO: what's left of that issue is getting solved.
Ten years ago you’d be right, but right now that business is dying rapidly.
TLSv1.3 on the other hand sometimes encrypts the hostname (eSNI) and most of the TLS handshake, so there's much less data to fingerprint. It's not as widely supported, but support is growing...
[1] https://engineering.salesforce.com/tls-fingerprinting-with-j...
//Edited to clarify that eSNI isn't default behaviour of 1.3
eSNI is not the default behavior, and has few deployments at scale. TLSv1.3 transmits SNI in the clear.
eSNI is being replaced with ECH[1], but in many cases, there is a 1:1 relation between the IP address and the site being served. ESNI and ECH are only one layer of obfuscation - a middleman (such as an ISP) could still snoop your DNS (unless DoH/DoT) and/or correlate the IP addresses you connect to against the hostname(s) presented on that server.
Attackers already do that today with nmap - scan publicly addressable ranges on port 443 and see what names are on the certificate presented by the server.
Encrypted Client Hello isn't finished. I would say the basic idea is settled, but there are plenty of technical nits and it might be next year before they have a final document.
Eventually the idea is that ECH will be GREASEd by always sending ECH data, if the client knows it is supported it will use ECH and if not then it will fill out the ECH data with random nonsense. Since it's encrypted, an adversary can't easily distinguish one from the other and a site which doesn't offer ECH will ignore the nonsense anyway.
The idea of probing servers on port 443 works well enough for dozens of popular sites with dedicated servers, but much less well for the long tail. A bulk host won't give you a list of every customer just because you hit port 443 on each server and pled ignorance, you'll get a generic "Under construction" page and no information.
That's supported in supported in tls 1.3, but actual deployment/usage is spotty (it's an extension, not mandatory). AFAIK it also requires your DNS to cooperate, since that's how it gets the keys for the initial handshake.
Wasn't there a HN story about people doing exactly that to figure out what condition people were looking up on WebMD? I don't recall when.
Depending on where you live the likelihood of your ISP doing something exceptionally nefarious might be way lower than some random VPN client someone finds on an appstore.
As garbage as most US ISP options are, I'd trust them long before I trust random VPN services. And I can be reasonably certain that my physical connection goes to Verizon. My virtual connection could be going anywhere and I just have to believe that it's to people who are who they say they are.
You’re just trading one for the other and that new one might not even have to follow the same laws.
any VPN worth its salt has a business model built around not logging data and not selling data. Your ISP on the other hand, is in the business of selling you internet access. Your data is a secondary revenue stream for them.
They two are not equivalent.
With us technical people it's more likely, but not necessary for others that may have just heard 'use a vpn' and went to the App Store, searched for 'vpn' and prepaid 3 years.
Hide my ass VPN is still up - https://www.hidemyass.com/en-us/index
Whether VPNs solve the issue or not is irrelevant to my point. Their primary advertised feature is to hide your traffic from your ISP, McDonalds or whoever, and people buy them. (Secondary feature is masking location for streaming services, which doesn't really work).