News for hackers. Stuff that matters.
Re:YouTube's road to HTTPS by coob on Monday August 01, 2016 @02:15PM (#12204625)
> We found that HTTPS improved quality of experience on most clients: by ensuring content integrity, we virtually eliminated many types of streaming errors

Wow, what were carriers doing to the streams?!

Re:YouTube's road to HTTPS by kbenson on Monday August 01, 2016 @02:22PM (#12204677)
It's long been known that some carriers purposefully degraded quality of youtube to keep from saturating their interconnections. That's the reason[1] behind the Google Video Quality Report[2]. Youtube does (did?) redirect you to a comparison of how well your ISP streamed content compared to others when there were streaming problems in some cases to combat this.

1: Well, I'm assuming, but it seems likely this was the main reason.

2: https://www.google.com/get/videoqualityreport/

Re:YouTube's road to HTTPS by lqdc13 on Monday August 01, 2016 @03:36PM (#12205247)
Degrading video quality is not a bad idea if you have limited bandwidth.

This would help in cases such as airplane flights. One person watching HD cat videos is going to consume more bandwidth than 20 people doing work.

Now, assuming bandwidth increase is difficult to achieve, they would be forced to keep increasing the price instead.

Prioritizing non-video content is challenging if all content is encrypted.

Re:YouTube's road to HTTPS by icebraining on Monday August 01, 2016 @03:40PM (#12205271)
Why not limit the bandwidth per user instead? Youtube itself will auto-adjust quality based on connection speed.
Re:YouTube's road to HTTPS by acdha on Monday August 01, 2016 @04:01PM (#12205434)
Exactly – this is both easier to implement and has the advantage of not incentivizing people to use something like a VPN to get more bandwidth at the expense of everyone else.
Re:YouTube's road to HTTPS by mdani on Monday August 01, 2016 @04:43PM (#12205775)
Per user limits are not easy to implement, maybe easier to visualize - consider that there are a bunch of packet gateways sitting behind a load balancer and each HTTP session may end up on a different server. There is no entity that counts the live bandwidth usage on a per user basis, let alone control it. Billing and metering is done on a session basis through logs. So from T-Mo's point of view it is much easier to detect a HTTP session as video and just throttle that session.
Re:YouTube's road to HTTPS by azernik on Monday August 01, 2016 @06:32PM (#12206579)
It is very easy to implement this (I have worked on such a limiter before). You pick the point of entry into your network (wi-fi connection, ISP connection), keep packet and byte counts for every such point of entry, and limit them.

ISPs have it especially easy, because they can be assured of being able to distinguish traffic from a given user (hardware control of the medium). It's a bit harder in wireless scenarios, since the client can spoof multiple different IDs, but it's hard for them to keep open a TCP connection under those conditions.

Re:YouTube's road to HTTPS by blowski on Monday August 01, 2016 @05:04PM (#12205971)
Eh? Even cheap home routers these days allow you to set QoS rules per device. Have I misunderstood something?
Re:YouTube's road to HTTPS by gleenn on Monday August 01, 2016 @07:40PM (#12206926)
Just because your cheap home router does it doesn't mean it scales to thousands of users on one router. Some home routers are actually quite capable AND very unsaturated. I'm not trying to defend carriers but it is a very apples to oranges comparison.
Re:YouTube's road to HTTPS by acdha on Monday August 01, 2016 @09:29PM (#12207302)
We did that on low-end PC hardware 15 years ago for conference and guest networks (800+ simultaneous active users, LAN and WiFi).

I find it unlikely that Linux, FreeBSD have gotten less efficient since then and the hardware has made enormous improvements, far in advance of the common uplink speed.

Re:YouTube's road to HTTPS by hueving on Monday August 01, 2016 @10:30PM (#12207502)
Are there thousands of people on a single flight?
Re:YouTube's road to HTTPS by dredmorbius on Wednesday August 03, 2016 @10:22PM (#12222962)
And you thought legroom was bad now....
Re:YouTube's road to HTTPS by warcode on Tuesday August 02, 2016 @04:12AM (#12208462)
Cheap home routers with QoS has OUTBOUND QoS.
Re:YouTube's road to HTTPS by mrb on Monday August 01, 2016 @09:24PM (#12207287)
Per-user limits is certainly way easier to implement than deploying a cluster of proxies doing on-the-fly transcoding of video streams(!)
Re:YouTube's road to HTTPS by TazeTSchnitzel on Monday August 01, 2016 @05:09PM (#12206012)
My university limits my bandwidth to 20mbps for HTTP/HTTPS and lower for SSH. Are they in a different position to normal ISPs?
Re:YouTube's road to HTTPS by foota on Monday August 01, 2016 @06:33PM (#12206580)
Really? T-Mobile should know your mac address and ip and be able to rate limit based on that.
Re:YouTube's road to HTTPS by lqdc13 on Monday August 01, 2016 @05:13PM (#12206045)
Let's say there are only 2 users using it. Then a lot of bandwidth would go unused if each user has a hard cap.

Ideally, there would be a dynamic limit based on network utilization and type of content consumed.

Re:YouTube's road to HTTPS by newman314 on Tuesday August 02, 2016 @02:46AM (#12208198)
Don't think hard cap. Think avail bandwidth/num of users.
Re:YouTube's road to HTTPS by spyder on Tuesday August 02, 2016 @05:27AM (#12208723)
The auto-adjustment is a problem for fast but traffic limited connections because it detects that the connection is fast, then it switches to HD or even 4K and the traffic limit gets used up faster even if the user don't need the better quality stream.
Re:YouTube's road to HTTPS by dyladan on Monday August 01, 2016 @03:48PM (#12205334)
Not sure about other services but at least GoGo was limited to about 500kbps per user last I heard.
Re:YouTube's road to HTTPS by _pmf_ on Tuesday August 02, 2016 @04:49AM (#12208569)
> Why not limit the bandwidth per user instead?

Because that would require actual effort from ISPs, which would be in conflict with their current business practice that can be summarized as "to us, you are all equally worthless".

Re:YouTube's road to HTTPS by aw3c2 on Monday August 01, 2016 @04:40PM (#12205749)
> One person watching HD cat videos is going to consume more bandwidth than 20 people doing work.

One HD stream probably uses more continuous bandwidth as those 20 people so just limit that. Allow bursts maybe.

> Prioritizing non-video content is challenging if all content is encrypted.

Good! It's none of their business what I am doing with my bandwidth!

Re:YouTube's road to HTTPS by kbenson on Monday August 01, 2016 @04:57PM (#12205890)
The problem here is from who's perspective is there "limited bandwidth"? To the ISP there is limited bandwidth available, but to the customer they are having the service they pay for purposefully degraded because the ISP doesn't want to deliver on what they've marketed.

> Now, assuming bandwidth increase is difficult to achieve, they would be forced to keep increasing the price instead.

And they should. That ISPs commonly use deceptive practices in pricing and delivering service is not a good thing. It reduces market information There should be much more granular pricing based on what's actually delivered, but the major ISPs don't want to go that way because then they would actually have to account for how what they deliver so often falls below what they market.

For example, when I moved into my brand new built house a couple years back and got a 25 Mbit Comcast connection set up, the following conversation happened:

Installer: Wow, you have the best signal I've every seen actually.

Me: Really? That's good. So what throughput am I seeing?

Installer: Let me check. (Installer does a speed/circuit test). About 14 Mbit.

Me: Didn't I order 25 Mbit?

Installer: Yes, but lots of things can affect that, such as line quality...

Me: (Having worked at a local ISP multiple times in the past for years, cuts him off, realizing the futility of this conversation). Okay, that's fine.

In what reality do "the best signal I've ever seen" and 56% of the advertised throughput coincide? (this was not because the connection was overused by others in the neighborhood either, it was fairly consistent at 14 Mbit).

> This would help in cases such as airplane flights. One person watching HD cat videos is going to consume more bandwidth than 20 people doing work.

So would separate tiers of connectivity. If you are doing business, you may be happy with a guaranteed minimum throughput, while other people (such as those streaming) might be fine to take up the slack or excess (since you can cache future video). We've had this for a long time through QoS.

> Prioritizing non-video content is challenging if all content is encrypted.

So don't prioritize based on content, prioritize based on connection.

Re:YouTube's road to HTTPS by semi-extrinsic on Monday August 01, 2016 @05:05PM (#12205979)
FWIW, every time (about 6 times in 3 years I think) I've complained about slow connections to my ISP, with documentation from speedtest.net that I'm getting less bandwidth than promised, they've given me that month free of charge.
Re:YouTube's road to HTTPS by zyxley on Monday August 01, 2016 @05:32PM (#12206195)
Sounds like there's some room here for a service that does speed tests and files complaints each month automatically.
Re:YouTube's road to HTTPS by semi-extrinsic on Monday August 01, 2016 @05:48PM (#12206318)
Yeah, I thought about it, but we recently got the decent-speed option included almost-free with our cable TV (15 mbps for $4/month), so I couldn't be bothered anymore. Used to be on 30 mbps for $40/month.
Re:YouTube's road to HTTPS by kevin_thibedeau on Monday August 01, 2016 @07:04PM (#12206749)
In which case you do what should have been done all along: Rate limit without regards to packet contents. People that still need better QoS than what that offers can pay for a higher tier to get a bigger chunk of the pie. With effective rate limiting in place the video bandwidth can be dropped by the server as necessary to maintain the stream. The infrastructure in the middle doesn't have any reason to meddle further.
Re:YouTube's road to HTTPS by Animats on Monday August 01, 2016 @08:26PM (#12207109)
It's turning out that HTTPS is more important for content integrity than for security. It's not surveillance that's the big problem for static content. It's middle boxes designed with the delusion they're entitled to mess with the content.
Re:YouTube's road to HTTPS by abalone on Monday August 01, 2016 @08:33PM (#12207135)
> It's not surveillance that's the big problem for static content.

Why would you say this? Surveilling what URLs someone is accessing / content they are watching / books they're checking out of the library has been a major security issue, historically.

Re:YouTube's road to HTTPS by Animats on Monday August 01, 2016 @09:01PM (#12207206)
HTTPS doesn't conceal the domain, or the length, or the order of requests. It's less of an issue for Google, because so much stuff comes from one domain. For small sites, figuring out who read what isn't a hard problem. In practice, you can probably buy that info from an ad tracking service.
Re:YouTube's road to HTTPS by abalone on Tuesday August 02, 2016 @04:51PM (#12213293)
You didn't qualify your claim as only pertaining to small sites. You simply said surveillance is not the big problem for static content. Given that most static content is served by large sites and the article is about Youtube, you haven't really supported that claim.
Re:YouTube's road to HTTPS by dredmorbius on Wednesday August 03, 2016 @10:27PM (#12222973)
I'd temper that statement significantly. Content integrity is the visible consequence of improved HTTPS compliance and use.

The biggest factor about surveillance is that you are rarely aware you're being surveilled. Direct evidence is rarely present.

Case in point, Michael Lewis's Flyboys. HFT trades intercepts weren't being overtly signalled, but were only evident when trades were structured such that they bypassed the opportunity to intercept intentions at the first market.

Re:YouTube's road to HTTPS by JBiserkov on Monday August 01, 2016 @02:20PM (#12204664)
Attempting to inject adds? Reduce the size by applying JPEG compression? :)
Re:YouTube's road to HTTPS by TD-Linux on Monday August 01, 2016 @02:43PM (#12204835)
When I had Sprint, they did in fact scale down and recompress all images in lower quality JPEG.

Some carriers even attempt to realtime transcode video into a lower bitrate. The implementations of this are universally poor and produce rather broken files.

Re:YouTube's road to HTTPS by dublinben on Monday August 01, 2016 @02:50PM (#12204897)
The more modern implementation of this (I'm looking at you, T-Mobile) is just to throttle all connections to content providers. They rely on the content providers' own adaptive streaming technology to deliver the appropriate 480p video when presented with a constrained connection.
Re:YouTube's road to HTTPS by mhurron on Monday August 01, 2016 @03:20PM (#12205135)
> The more modern implementation of this (I'm looking at you, T-Mobile) is just to throttle all connections to content providers.

If YouTube can't handle traffic shaping, Youtube is broken.

Re:YouTube's road to HTTPS by dublinben on Monday August 01, 2016 @03:46PM (#12205318)
If your ISP applies artificial traffic shaping, your ISP is broken.
Re:YouTube's road to HTTPS by amscanne on Monday August 01, 2016 @04:44PM (#12205784)
What's "artificial" shaping?

The average consumer ISP has an over subscription ratio of 70-to-1.

Unless you want to pay 70x as much for your "100mbit" connection, there are going to be times when packets get dropped.

Isn't it better to drop packets fairly among subscribers? No shaping would result in whoever is using the most dominating everyone else.

This basic principle is still neutral; you don't have to shape based on destination / content provider.

Re:YouTube's road to HTTPS by rhino369 on Monday August 01, 2016 @03:52PM (#12205370)
I should be able to turn it off, but I like that tmo throttles my video. I only get 2.5 gb, I don't need 1080p.
Re:YouTube's road to HTTPS by mahyarm on Monday August 01, 2016 @05:03PM (#12205957)
You can turn it off, but it removes the zero rating. Up to you either way.
Re:YouTube's road to HTTPS by rhino369 on Tuesday August 02, 2016 @12:50AM (#12207898)
I'm on the light package so I don't even get the zero rating.
Re:YouTube's road to HTTPS by mhurron on Monday August 01, 2016 @05:06PM (#12205988)
What the ISP is doing isn't any different than what any private network could do. If YouTube can not handle that, Youtube is not playing well with standard network practices and is broken at a technical level.

Your ISP doing traffic shaping is a question of if they should, not a technical question.

Re:YouTube's road to HTTPS by bisby on Tuesday August 02, 2016 @10:03AM (#12209975)
In this case, that's what "broken" means. Not that there is something technically wrong, but that they are doing things that they shouldn't.
Re:YouTube's road to HTTPS by azernik on Monday August 01, 2016 @06:34PM (#12206590)
That shaping should not be broken by HTTPS. The only things that would be affected by HTTPS is carriers munging the bits.
Re:YouTube's road to HTTPS by ianlevesque on Monday August 01, 2016 @03:20PM (#12205132)
Not ideal but that's a lot less intrusive.
Re:YouTube's road to HTTPS by acdha on Monday August 01, 2016 @04:05PM (#12205464)
The most common implementation of this appears to be from a company called Bytemobile which also injects some pretty buggy JavaScript into HTML content.

The good news is that you can disable all of their degradation by adding "no-transform" to your Cache-Control headers:

https://kornel.ski/en/proxies

“The standard HTTP header Cache-Control: no-transform which tells proxies not to modify responses was respected by all proxies I've encountered. This is amazing! It wins an award in the "Best Supported Obscure Header" category.”

Re:YouTube's road to HTTPS by chakalakasp on Monday August 01, 2016 @03:00PM (#12204980)
I used to think it was silly to use a VPN on a cellphone not on WiFi. Not these days, not so much.
Re:YouTube's road to HTTPS by mholt on Monday August 01, 2016 @03:03PM (#12205010)
I grew up in rural Iowa with a poor Internet connection (satellite) that dropped packets if there were thick clouds, any snow or rain, even wind sometimes. Integrity checks would have helped a lot.
Re:YouTube's road to HTTPS by mikeash on Monday August 01, 2016 @03:22PM (#12205142)
TCP already covers that sort of thing. What HTTPS adds is integrity checks against an intelligent attacker, not just random natural interference.
Re:YouTube's road to HTTPS by jasonuhl on Monday August 01, 2016 @04:38PM (#12205725)
TCP only gives you 16 bits of checksum, which is really not enough. I used to work for a CDN which served downloads over HTTP, and when the object size grew to a few gigabytes a non-negligible percentage of users ended up with corruption.

The SSL MAC is valuable even in the absence of enemy action.

Re:YouTube's road to HTTPS by DanielDent on Monday August 01, 2016 @10:28PM (#12207497)
This is a really interesting 'real world' anecdote. Do you know of any related data available publicly? i.e. "on our network, X bits in Y terabytes end up with undetected corruption, meaning that approximately Z% of downloads of a 2GB file will have at least one bit error".
Re:YouTube's road to HTTPS by Dolores12 on Tuesday August 02, 2016 @12:40AM (#12207860)
From http://noahdavids.org/self_published/CRC_and_checksum.html

In "Performance of Checksums and CRCs over Real Data" Stone and Partridge estimated that between 1 in 16 million and 1 in 10 billion TCP segments will have corrupt data and a correct TCP checksum. This estimate is based on their analysis of TCP segments with invalid checksums taken from several very different types of networks. The wide range of the estimate reflects the wide range of traffic patterns and hardware in those networks. One in 10 billion sounds like a lot until you realize that 10 billion maximum length Ethernet frames (1526 bytes including Ethernet Preamble) can be sent in a little over 33.91 hours on a gigabit network (10 * 10^9 * 1526 * 8 / 10^9 / 60 / 60 = 33.91 hours), or about 26 days over a T3.

Re:YouTube's road to HTTPS by mikeash on Monday August 01, 2016 @07:18PM (#12206823)
Makes sense. Thanks for the informed correction.
Re:YouTube's road to HTTPS by jonathanoliver on Monday August 01, 2016 @04:51PM (#12205829)
The worst is when you hit bandwidth caps and they cut you off. Gotta love background downloads...
Re:YouTube's road to HTTPS by mholt on Monday August 01, 2016 @05:03PM (#12205958)
Remember those "download accelerator" programs? Even when we had dial-up (no weather interference over the air), we had corrupted downloads quite often.
Re:YouTube's road to HTTPS by drzaiusapelord on Monday August 01, 2016 @03:23PM (#12205154)
I imagine this was a snarky comment about services like T-mobile's binge. They limit video 720p or 480i at a moderate bitrate for mobile. Its supposed to be opt-in but I wouldn't be surprised if many mobile providers peppered the video stream with errors to knock down costly high-bitrate 1080p streams.
Re:YouTube's road to HTTPS by dawnerd on Monday August 01, 2016 @05:46PM (#12206308)
Opt in? It was opt out unless it changed very recently. I had a call with their corp office to plea why my plex server should also qualify but they said it wouldn't.
Re:YouTube's road to HTTPS by Thaxll on Monday August 01, 2016 @03:34PM (#12205225)
Remind me the madness that mobile carrier are doing to the TCP stack.
Re:YouTube's road to HTTPS by mh- on Tuesday August 02, 2016 @06:01AM (#12208818)
any links with technical details about this?