As for SCTP, I think applications generally drive progress better than OSes do. Certainly the OSes have had enough time to sort out SCTP, but they haven't done it, and the end-game is nowhere in sight. Until there is an app that showcases it, will it ever happen?
My recommendation is to first prove new protocols at the application layer (perhaps run SCTP over UDP just to grease the skids). When people are addicted to how much better the experience is (multihoming + concurrent streams), then the OSes will pull it into the kernel to make it more scalable.
What do you think?
SPDY is always an upgrade; AFAIK there's no way to start out talking SPDY (since there's no spdy: scheme). Every SPDY server should still support old-fashioned HTTP.
SCTP gets broken by NATs, so outside of a few telco networks it's dead.
I fundamentally believe the protocol has to be server authenticated and encrypted always. We've seen breach after breach of user privacy and the desire for governments to crack down on its citizens has no bound. We simply have to encrypt. Once we do that, text is off the table, and it doesn't really matter if it is binary or not.
Of course, we need much better tools for managing the encryption, I'm not a believer that the status quo is adequate in any way.
This is exactly why I wrote "high-level protocols". With just a telnet client I can talk HTTP, FTP, SMTP, IMAP, POP3, IRC and so on. And the machine just needs to handle lowel-level details.
Now if only TCP was message oriented... For that, I guess we have ZMQ.
It could be argued that there is no "need" for this, but I find myself doing it quite frequently all the same. That may have something to do with the fact that I am developing a relatively low-level reverse HTTP proxy tool (PageKite), but I also did it quite a bit when working on more traditional webapps.
Once you speak the protocol it's nice to be able to just connect and chat with the server directly. :-) Humans are really bad at speaking binary.
Well, I've did so when I had to debug some redirect caching problem with "walled garden" setup. This could also be done with a more specialized software, though, I just happened to use telnet as it was readily available.
I also sometimes check server's availability (and overall response correctnes) by telnetting to it. Although, I was just too lazy to set up a tunnel to the LAN they're residing at.
Take Arduino for example, writing a naive but working http client is childs play; writing a SPDY client in Arduino is well beyond many people's capabilities. Yes libraries will appear eventually, but we've not even got a decent open http library yet.
Ancient people — even though they hadn't much CPU, memory and network resources — in all their wisdom made high-level protocols text-oriented. Sure, they didn't consider everything (so we're still haunted with evil spirits of, for example, UTF-7 and quoted-printable), but overall design was simple yet perfectly functional. And anyone could use just a telnet client to talk to HTTP (FTP, SMTP, etc) server. With SPDY you can't do such thing anymore, you need specialized software.
I'm all for something like "Upgrade: SPDY"/"HTTP/1.1 101 Switching Protocols" (then binary framed message-oriented protocol goes into effect), but I somehow worried with binary-from-the-beginnig "HTTP 2.0". Maybe I'm just too conservative?
(Side question: If we want multiplexing streams so badly, why noone cares about SCTP? Is it flawed or inappropriate?)