News for hackers. Stuff that matters.
Re:Netflix now supports TLS 1.3 by saurik on Tuesday April 21, 2020 @08:15AM (#22934212)
> If I'm watching a show, I don't necessarily want every single router along the way knowing what show i'm watching.

Unless Netflix has also started padding their compressed video segments to a deterministic size, TLS doesn't really help with the "I don't want people to know what video I'm watching" problem, as the encrypted media segments are first compressed using a variable bitrate encoding that causes a unique pattern in their resulting chunk sizes; and so, with a little database of signatures--apparently even just at the TCP level, without caring at all about the HTTP and TLS layers... which surprised even me, and I'm quite cynical about this stuff--you can pretty rapidly fingerprint what movie someone is watching from a passive traffic dump.

https://dl.acm.org/doi/10.1145/3029806.3029821

> Identifying HTTPS-Protected Netflix Videos in Real-Time (2017)

> After more than a year of research and development, Netflix recently upgraded their infrastructure to provide HTTPS encryption of video streams in order to protect the privacy of their viewers. Despite this upgrade, we demonstrate that it is possible to accurately identify Netflix videos from passive traffic capture in real-time with very limited hardware requirements. Specifically, we developed a system that can report the Netflix video being delivered by a TCP connection using only the information provided by TCP/IP headers.

> To support our analysis, we created a fingerprint database comprised of 42,027 Netflix videos. Given this collection of fingerprints, we show that our system can differentiate between videos with greater than 99.99% accuracy. Moreover, when tested against 200 random 20-minute video streams, our system identified 99.5% of the videos with the majority of the identifications occurring less than two and a half minutes into the video stream.

Note that this exact same kind of attack has been applied successfully to other places where a pattern of sizes can differentiate usage, including: Google Maps tiles (which are either variably-compressed fixed-dimension images or variably-dense fixed-region vectors), web pages (which are different sizes already but also link to images and other resources that are random sizes, which helps create the fingerprint), and non-personalized type-head search queries (where the result for each letter brings up some unknown set of titles and URLs; as you type each character the sequence of sizes for your intermediate results can expose the query, potentially even without needing any model of the likely keywords).

Maps: https://ioactive.com/ssl-traffic-analysis-on-google-maps/

Search: https://eprint.iacr.org/2014/959.pdf

(edit: I just realized it might be interesting for me to note that I am the lead developer of a new VPN protocol called Orchid and have "attempting to solve as many of these issues as generically and yet with as little required overhead as I can" on my todo list; to be very clear: I don't handle any of this stuff yet at all, but should get to it in the next few months... while the company behind Orchid that is paying me to work on the protocol talks a lot about itself already, I'm really intending to do my own "launch" of the product once I solve some of the stability issues and finish my traffic analysis mitigations.)

Re:Netflix now supports TLS 1.3 by tialaramex on Tuesday April 21, 2020 @08:44AM (#22934423)
Yup. One thing that I'm surprised to see hasn't happened already, but maybe has a better chance in QUIC than TLS 1.3 itself is making use of TLS 1.3 padding to exactly fill the MTU on typical 1500 byte (or so) links.

The TLS 1.3 on-wire format is weird because it needs to pass rusted-in-place TLS 1.2 only middleboxes. One of the convenient side effects is that you can add one or more bytes of padding "free". Any 1499 byte packet can be transformed into a 1500 byte packet just by adding one byte of padding whereas many naive formats couldn't do that.

So this means you could "right size" packets just before sending, so that you still send the same number of packets but they're always full, reducing the ability to distinguish their contents by length.

Signal does something more sophisticated to defeat this sort of analysis for its GIF support. The local client does two or more overlapping GETs. So maybe it needs a 14230 byte GIF but it fetches 8430 and 9401 bytes of data then throws some of it away to re-assemble the 14230 bytes needed, an adversary is left with too many possibilities. But it's probably a bit much to expect Netflix subscribers to pay extra for bandwidth just to defeat an adversary who wants to know how much of Tiger King they've seen.

Re:Netflix now supports TLS 1.3 by toast0 on Tuesday April 21, 2020 @10:30PM (#22941441)
Netflix's aproach of doing TLS in the (FreeBSD) kernel may make it easier to do this. In most stacks, the TLS segmentation is too far away from the TCP segmentation to make this practical, especially since MTU/MSS can change during a connection.
Re:Netflix now supports TLS 1.3 by Klathmon on Tuesday April 21, 2020 @08:28AM (#22934308)
That's really interesting, if i'm understanding this correctly it's basically another form of compression leaking information about the underlying data. Would they need to pad everything to a deterministic size or would they just need to change some early parameters during the encoding process to throw off the rest of the chain?

But in the end it still doesn't mean TLS isn't worth it for all the other benefits (not that you implied that).

Re:Netflix now supports TLS 1.3 by saurik on Tuesday April 21, 2020 @08:45AM (#22934430)
The way static "streaming" video works is that you divide the video into a number of small segments--maybe every 10 seconds or every 2 seconds or something--and then encode each of those segments at various quality levels and store all of those files on HTTP servers. There is then a "manifest" file that lets the client learn what qualities are available and what their URLs will be for various timestamps. The client then just starts downloading chunks. If it is finding itself with free time, it starts downloading higher quality (larger) chunks in the future. If it isn't downloading them fast enough, it downloads lower quality (smaller) chunks in the future.

The issue then is that no matter what you do with the compression algorithms, the user is downloading these chunks and you can see the pattern of packets in one direction setting up requests and the packets in the other direction replying with the chunks and you can fingerprint what movie someone is watching. It isn't some kind of encoding issue, as the data is all encrypted: it is the entire concept of taking fixed length segments of a movie that will compress to some non-determinstic size. If you take the first two minutes of every Star Wars movie, divide each up into 10 second segments, and then compress those segments, the sequence of sizes of the segments will be pretty unique.

What is so great about this particular paper is that you don't even really need to analyze the TLS layer and try to pay close attention to really figure out the request/responses: they just fingerprint the TCP flow and that's sufficient, which in retrospect doesn't surprise me as what you are really looking for is some kind of rate of requests to responses for the chunks over the course of those first few minutes of watching the video, and don't really need to know for sure where the boundaries are: you have a long enough sequence and a small enough catalog (there aren't tens of millions or billions of videos on Netflix) to get a really strong fingerprint using just the relative rates.

To fix this you really need to either inject extra random traffic (such as extra packets to the server that break up the request rates) that adds so much noise that you can't figure out the signal "in time"--if it takes longer to fingerprint a movie than the length of a typical movie, that's "good enough"--or you need to destroy signal (which is a better description of what we do if the video segments are all the same size: at that point all movies are by definition the same sequence over and over again; if you then pad the length of every movie to the same 4 hour runtime and force the user to download all of the padded black video frames, you essentially 100% solve the problem ;P).

Re:Netflix now supports TLS 1.3 by SketchySeaBeast on Tuesday April 21, 2020 @09:45AM (#22934910)
Doesn't this inspection of the manifest have a fairly limited scope in how it can be exploited though? I know the attacker will be able to see what someone is watching, but only for non-novel information, right? You can't use that technique to guess anything new, and had to have yourself already seen the video?
Re:Netflix now supports TLS 1.3 by saurik on Tuesday April 21, 2020 @09:55AM (#22934994)
What I am talking about hers is definitely focussed only on "fingerprint known content", as that is the goal of the "figure out which Netflix movie I am watching" use case (as well as the "what model am I watching on myFreeCams/Chaturbate" use case, which might feel worse ;P... it is also the "what video am I watching on PornHub" use case, but that's inherently harder as there are so many many videos and they are often super short: much harder than the moderate catalog of long movies and tv shows on Netflix or the comparatively scant number of available live feed on camsites).

That said, as noted by someone else on this thread, there has been some work done figuring out what people are saying by analyzing encrypted speech packets; but I imagine that kind of technique would be almost impossible to pull off with these segments on the order of multiple seconds long, and including the video in the stream would seem to make that a total non-starter.

Re:Netflix now supports TLS 1.3 by tialaramex on Tuesday April 21, 2020 @08:58PM (#22940969)
Well one of the things that paper does is they just go fetch the index.

Every Netflix client that is watching say 1080p Rick & Morty s1e2 fetches an index of the bits that make up that video, and then fetches the relevant chunks of data.

So if you just fetch that index, no need to "watch" the video, you know OK 1390450 bytes is the 1st chunk of data making up that video, and then 2046917 bytes is the 2nd chunk. When you see that somebody fetched 1390450 bytes of encrypted data from Netflix, they're probably watching s1e2 of Rick & Morty in 1080p. When they fetch 2046917 bytes of encrypted data that confirms it.

So it's "non-novel" but they don't need to (relatively expensively) actually watch the videos they want to match. It's likely any sophisticated adversary has an index of all the common videos from sites like Netflix, the Play Store, Disney+ and so on.

There is definitely more a client could do to make this harder, but there'd be a sharp trade-off where to make it impossible for passive eavesdroppers to know what you saw costs both you and the "broadcaster" a lot of bandwidth, and bandwidth ain't free.

I'd like to see Netflix at least gently stroll in that direction, maybe use some TLS padding to make it a bit harder to do this trick, but I don't expect them to set off at a sprint and ruin their profitability in pursuit of a non-goal.

Re:Netflix now supports TLS 1.3 by MaxBarraclough on Wednesday April 22, 2020 @04:35AM (#22943446)
> there'd be a sharp trade-off where to make it impossible for passive eavesdroppers to know what you saw costs both you and the "broadcaster" a lot of bandwidth, and bandwidth ain't free

Off the top of my head: if you took a live streaming approach along the lines of live TV, or indeed Google Stadia/GeForce Now/OnLive, then you could presumably make the traffic patterns invariant over what was being watched. You'd need to ensure the bitrate didn't drop during low-detail scenes, I suppose, similar to what you mentioned re. Skype below.

That approach would of course be far more intensive on server resources, but I'm sure a similar level of privacy could be achieved without such inefficiency.

Re:Netflix now supports TLS 1.3 by IanCal on Tuesday April 21, 2020 @08:39AM (#22934384)
VOIP compression of speech too, at least in the past: http://www.cs.unc.edu/~amw/resources/hooktonfoniks.pdf

edit - said Skype initially but not sure it was specifically skype

Re:Netflix now supports TLS 1.3 by tialaramex on Tuesday April 21, 2020 @08:51AM (#22934470)
You can defeat this for VoIP by using a codec in CBR (Constant Bitrate) mode. You should look for this feature in secure audio software.

Unfortunately for video CBR is generally thought to be too expensive to be acceptable. 50kbps CBR Opus wastes 22Mbytes of data in an hour if you're silent compared to a good VBR codec. A small price to pay for improved security. But a CBR Netflix encoding might move several gigabytes per hour of extra data compared to a visually similar VBR encoding, and that's going to really hurt both Netflix and quota restricted customers.

Re:Netflix now supports TLS 1.3 by merb on Tuesday April 21, 2020 @08:33AM (#22934344)
well your first attack is only theoretically possible. but it's basically impossible in a big network.

for a single user it's possible. for a network with gigabytes of data, it's basically impossible. it's not just a "tcpdump" and compare against fingerprint.

btw. it's also outdated.

Re:Netflix now supports TLS 1.3 by saurik on Tuesday April 21, 2020 @08:47AM (#22934438)
Why would the size of the network possibly matter?... You do know that you can just filter the packets by IP address, right? ;P
Re:Netflix now supports TLS 1.3 by merb on Tuesday April 21, 2020 @08:54AM (#22934494)
yeah with a targeted attack. but I've meant big scale monitoring which person watches what.

also eme basically solves this on the audo and video side. (maybe not the best solution tough)

Re:Netflix now supports TLS 1.3 by Aissen on Tuesday April 21, 2020 @11:29AM (#22935776)
I think you overestimate how complex this is to purpose-build once you have the basics of DPI in place.

Also you don't need a precise measurement. Just measuring a random 10k users in 1M network daily will give you more precise viewership measurement than anything Netflix publishes.

Re:Netflix now supports TLS 1.3 by Aissen on Wednesday April 22, 2020 @05:53AM (#22943913)
Replying to self… I'm wondering if the recent per-country daily "Top 10" that Netflix is now exposing in its UI (now scraped by https://flixpatrol.com/) is not a response to ISPs already building this ? Like a way to kill two birds with one stone: reduce value of private measurements, and improve UX at the same time.
Re:Netflix now supports TLS 1.3 by MaxBarraclough on Tuesday April 21, 2020 @09:29AM (#22934759)
How does EME change anything?
Re:Netflix now supports TLS 1.3 by merb on Wednesday April 22, 2020 @03:46AM (#22943201)
EME encodes the video content. i.e. without eme you can basically extract the video via tcpdump, with eme you can't since everything, even metadata is encrypted.

the attack inside the pdf does basically mention the old technology which netflix used and not dash+eme inside video element (MPEG-CENC). i.e. widevine which is kinda funny because the browser drm is highly controversial, but in this case serves as additional privacy since video metadata and content get's encrypted, no matter which transport layer is used.

Re:Netflix now supports TLS 1.3 by MaxBarraclough on Wednesday April 22, 2020 @12:43PM (#22947490)
> EME encodes the video content

EME encrypts the video content, yes, but I don't see that this changes things. Netflix uses HTTPS (TLS) encryption over the top of the EME encryption, as this thread's title states.

> even metadata is encrypted.

I don't believe so. If you're delivering EME-encrypted blobs over insecure HTTP, an ISP will be able to see which blobs you are requesting, simply by their URL.

Aside: I recall reading that Netflix's CDN servers ('OCAs') store EME-encrypted blobs, so only the HTTPS encryption burdens the delivery server's CPU. Unsurprising, of course.

> without eme you can basically extract the video via tcpdump

Six years ago, sure. Today, no, as the stream is sent over HTTPS.

> in this case serves as additional privacy since video metadata and content get's encrypted, no matter which transport layer is used.

Perhaps unencrypted streams are still used to support legacy devices, but Netflix are committed to maximal use of HTTPS.

I don't see how EME's additional layer of encryption changes anything as far as privacy and unscrupulous ISPs are concerned.