You can choose to start out with skepticism or cynicism if you like but there’s no defect in people who don’t.

Its nothing about faith here. I was looking at the evidence. The problem was that a lot of people I was working with had faith that SPDY was something that it wasn't.

They either interpreted what I was saying as either an excuse to not change infra (the change was minimal), "old think" because I was not a hot young fullstack node engineer, or that they thought I was trying to be more clever than google.

What seems more likely?

The main problem you're talking of is head of line blocking due to packet loss. But packet loss as a congestion signal is nowhere near as common as people think, and that was already the case during the original SPDY design window. End-user networks had mostly been set up to buffer rather than drop. (I know this for an absolute fact, because I spent that same time window building and deploying TCP optimization middleboxes to mobile networks. And we had to pivot hard on the technical direction due to how rare packet loss turned out to be on the typical mobile network). The real problem with networks of the time was high and variable latency, which was a major problem for H1 (due to no pipelining) even with the typical use of six concurrent connections to the same site.

Second, what you're missing is that even a marginal improvement would have been worth massive amounts of money to the company doing the protocol design and implementation. (Google knows exactly how much revenue every millisecond of extra search latency costs). So your "marginally better" cut isn't anywhere near as incisive as you think. It also cuts the other way: if SPDY really had been making those metrics worse like one would expect from your initial claims about H2 performing worse than H1, it would not have launched. It would not have mattered one bit whether the team designing the protocol wanted it deployed for selfish reasons, they would not have gotten launch approval for something costing tens or hundreds of millions in lost revenue due to worse service.

Third, you're only concerned with the downside of H2. In particular HPACK compression of requests was a really big deal given the asymmetrically slow uplink connections of the time, and fundamentally depends on multiplexing all the requests over the same connection. So then it's a tradeoff: when deciding whether to multiplex all the traffic over a single connection, is the higher level of vulnerability to packet loss (in terms of head of line blocking, impact on the TCP congestion window) worth the benefits (HPACK, only needing to do one TLS handshake)?

It didn't need to be the Holy Grail to be worth creating. It just needed to be better than H1 was, or better than what H1 could be evolved to with the same amount of effort. And likewise, it's totally possible for H2 to be better than H1 while also H3 is better than H2.

You appear to be confused by the idea that somebody would ever create something other than the absolute best possible thing. Why create H2 first, rather than just jump straight to H3?

One obvious reason is that H2 is far less complex than H3 thanks to piggybacking on TCP as the transport mechanism. The downside is that you then have to deal with all the problems of TCP as well. At the time, there would have been a lot of optimism about eliminating those problems by evolving TCP. An extension for connection migration, an extension for 0-RTT connection setup, an extension for useful out of order data delivery, etc.

It was only a bit later that it became clear just how ossified TCP was. Up until then, one could have told a story about how Microsoft controlled the primary client operating system, and were not really motivated to implement the RFCs in a timely manner, and that's why the feature rollouts were taking a decade. In the 2010s, it became clear that evolution was impossible even when all the coordination and motivation was there. See TCP Fast Open for a practical example.

So around 2015-ish you see big tech companies switch to UDP-based protocols just so that they can actually do protocol evolution.

The other plausible reason is that it's going to be far easier to get the initial work for making H2 funded, since the scope is more limited. And once you show the real-world gains (which, again, would have been there since H2 is better than H1), you then have the credibility to get funding for a more ambitious project.

I agree that H2 is defacto better than H1, and easier to implement when compared to H3. However, I'll call out the 2 biggest time sinks of the RFC: stream prioritisation and server push. Both of which had narrow application, and incomplete/inefficient specification. H3 seems to have ditched both. My question is, how did this ever end up in the final RFC? As both seem like the kind of thing that could have been easily disproved in SPDY deployments, as well as just asking people doing HTTP for a living.

It has almost _always_ been tied to OS, but moreover the OS of every node in between you and the webpage. That was the most frustrating thing, there were and are solutions for making TCP more latency resistant but also get better throughput and deal with "buffer bloat" which was a big thing at the time.

I was working in transcontental bulk transfer which used to mean that things like aspera/fasp was the defacto standard for fast/latency/loss resistant transport. So I had seen this first hand. I suspect it was probably why I was dismissed, because I wasn't a dyed in the wool webdev.

Surely packet loss due to poor signal quality is rather common over mobile networks and that packet loss still affects TCP's congestion window.

Admittedly anecdotal, but I just connected to a 5G network with low signal strength and it certainly seems to be the case.

The base stations and terminals are constantly monitoring the signal quality and adjusting the error correction rate. A bad signal will mean that there's more error correction overhead, and that's why the connection is slower overall.

Second,the radio protocol doesn't treat data transmissions as a one-and-done deal. The protocols are built on the assumption of a high rate of unrecoverable transmission errors. Those rates would be way too high for TCP to be able to function, so retransmissions are instead baked in at the physical protocol level. The cellular base station will basically buffer the data until it's been reliably delivered or until the client moves to a different cell.

And crucially, not only is the physical protocol reliable but it's also in-sequences. A packet that wasn't received successfully shows up just as one (radio) roundtrip latency blip during which no data at all shows up at the client, not as packet loss or reordering that would be visible to the TCP stack on either side.

Other error cases you'd get are:

- Massive amounts of queueing, with the queues being per-user rather than per-base station (we measured up to a minute of queueing in testing). The queue times would translate directly to latency, completely dominating any other component.

- A catastrophic loss of all in-flight packets at once, which we believed was generally caused by handover from one cell to another.

If you don't understand something, it'd be more productive to ask questions than just make shit up like that.

In principle you could observe it from some kind of OS level counters that aren't ACLd, but in practice the counters are not reliable enough for that.

Now, the things like "calls dropping" or "connections getting reset" that you're calling out have nothing to do with packet loss. It's pretty obvious that you're not very technical and think that all error conditions are just the same thing and you can just mix them together. But what comes out is just technobabble.

I'm saying that from the point of view of either endpoint, there is very little packet loss in mobile networks, because of error correction and retransmissions being handled at the physical layer. This is the third time I've written it. Both previous times you've not answered that, and instead made up a strawman about HTTP/2 and magic. Why do you keep doing that?

Do you not believe that cellular radio protocols do error correction? Or that they do retransmissions at that level, rather than just try transmitting each packet once and then give up?

Two points:

1. It's not commonly realised the TCP is terrible on lossy networks, where terrible means gets less than 10% of the potential throughput. It only becomes apparent when you try to use TCP over a lossy network of course, and most real networks we use aren't lossy. Engineers who try to use TCP over lossy networks end up replacing it with something else. FWIW, the problem is TCP uses packet loss as a congestion signal. It handles congestion pretty well by backing off. But packet loss can also mean the packet was actually lost. The right responses in that case are to reduce the packet size and/or increase error correction, but _not_ decrease your transmission rate. Thus two responses to the same signal conflict.

2. Because of that, the layer two networks the internet uses have evolved to have really low error rates, which is why most people don't experience TCP's problems in that area. As it happens just about any sort of wireless has really high error rates, so they have to mask it. And they do, by having lots of ECC and doing their own ACK/NAKs. This might create lots of fluctuations in available bandwidth - but that is what TCP is good at handling.

By the by, another reason we have come to depend on really low error rates on layer 2. That's because TCP's error detection is poor. It lets roughly one bad packet through in every 10,000. (Adler32 is very poor on small packets.) You can send 100,000 packets a second at 1Gb/sec, so you need to keep the underlying error rate very low to ensure the backup you are sending to Backblaze isn't mysteriously corrupted a few times a year. <rant>IMO, we should have switched to 64 bit CRC's decades ago.</rant>

> -these highly experienced guys actually knew what they were doing

> What seems more likely?

Well, when you put it that way... the former. By a large margin.

You're making quite clear you are the type of person who is extremely quick to accuse everyone and anyone of being incompetent in the absence of evidence or in spite of evidence.

You do not need to Google too hard to find tons of open-source benchmarks of real world servers showing off performance gains from switching to HTTP/2 and HTTP/3.

But here you are, claiming everyone is incompetent and that their work was bad. In spite of all the evidence.

It's clear that you have nothing relevant to say about the topic and no evidence to even suggest your beliefs have a leg to stand on.

These junior developers also have a tendency to make improvements to the system by implementing brand-new features instead of improving old ones. Look at recent Microsoft releases: we don't fix old features, but accrete new ones. New features help much more at review time than improvements to old ones.

(That's literally the explanation for PowerShell. Many of us wanted to improve cmd.exe, but couldn't.)

[0] https://blog.zorinaq.com/i-contribute-to-the-windows-kernel-...

(I'd heard the story in person from Snover years before that, so it definitely wasn't just sour grapes from somebody about to leave the company.)

I can totally believe that MS had a culture where making improvements to cmd.exe was impossible to justify, but that doesn't actually mean that a greenfield project in the same space would be considered promo-worthy either. It seems more likely that everything in that space was viewed as career suicide and you needed to be working on Internet stuff, just like at today's bigco anything except AI will have the same problem.

Isn't Windows Console the (old) terminal emulator, with cmd.exe being just the command-line interpreter.

Before that, csrss.exe handled terminal emulation (which is why console windows weren't themable[1]).

[1]: https://devblogs.microsoft.com/oldnewthing/20071231-00/?p=23...

How do you explain that some HTTP/2 server implementations handle an order of magnitude more connections than their HTTP/1.1 implementations? Is this something you downplay as accidental benefits of doing something to propel their career?

https://http2benchmark.org/

You should talk less about conspiratorial accusations you are making up on the spot and more about hard data.

As someone who was part of the rollout of HTTP2 for a $large_website, I can confirm that "this will harm mobile performance" was outright and flatly rejected. This included people who were our reps on the w3c. I just had to sit there and wait for the real world metrics to come in

"multiplexing will remove bottlenecks!"

"benchmarks prove that its faster!"

"you just don't understand how TCP works"

"the people at google are very smart, what do you know?"

"server push will reduce latency"

etc etc etc.

We even had the graphs of page size over time (going ever up) and average usable bandwidth (not keeping up, especially on mobile) None of that mattered until the rollout had a real world effect on our performance.

This sounds like a bullshit conspiratorial excuse. If you have real world data and you aren't afraid of having peers looking through it, nothing prevents you from presenting it to peers.

So where is that data?

Instead, you just have vague unsupported unbelievable claims made by random people in the internet, as if that's any way to decide over policy, and any faint doubt raised over that claim is faced with conspiratorial remarks complemented by statements on how everyone around OP is incompetent except him.

I will go as far as to claim OP's assertion is unbelievable, to the point of sounding like bullshit. It's entirely unbelievable that people designing protocols for a multinational corporation whose bread and butter is stuff done over TCP connections were oblivious to how TCP works, and the most incompetent of them would bother to design the first major revision of HTTP. Unbelievable.

But hey, some random guy online said something, so it must be true!

If that article really exists,if you read it, if you paid attention to it,and if you really felt it was relevant, you'd already have posted the link.

But no. You're just throwing distractions on how others can Google for stuff.

Enough.

If you don't want to be banned, you're welcome to email [email protected] and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.

If you don't want to be banned, you're welcome to email [email protected] and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.

https://bugzilla.mozilla.org/show_bug.cgi?id=779413

In their performance tests vs HTTP 1.1 the team simulated loading many top websites, but presumably by accident used a single TCP connection for SPDY across the entire test suite (this was visible in their screenshots of Chrome's network panel, no connection time for SPDY).

They also never tested SPDY against pipelining - but Microsoft did and found pipelining performed the same. SPDY's benefit was merely a cleaner, less messy equivalent of pipelining.

So I think it's fair to say these developers were not the best Google had to offer.