Ah, the old middleboxes excuse. Changing TCP would break the middleboxes, so instead we'll invent a brand new protocol which is (theoretically) ignored by the middleboxes. The idea being if you slip something in via UDP, nobody will notice, and then you encrypt it so the middleboxes give up on it.

This assumes that lack of transparency is the only way forward. I'd argue that IP and TCP ossified not because of transparency, but because the design didn't make it easy enough to simultaneously remain compatible while adding new functionality. Yet we know a transparent, grokable, mungeable, backwards-and-forwards-compatible protocol that plays well with middle boxes is doable: HTTP. We could design lower-level protocols to have the same design properties, with both long-lived backwards compatibility and the ability to adopt new functionality. If the problem is with middleboxes, then the solution shouldn't be "how can we avoid them", but "how can we make a protocol that works well with them".

(As a side note, a lack of transparency has chilling effects. Since the adoption of TLS v1.3, more and more TLS security has been broken in order to provide the necessary IT functions of traffic inspection, caching, routing, classifying, shaping, and blocking. Nation states will need to continue creating their own fragmented walled gardens, and continue invading privacy more, precisely because we're giving them less control over how they manage these functions which are a matter of national policy. You can't software you way out of a process problem.)

You're arguing for the "smart network" approach. This approach already failed. Some HN readers might not even have been born when that happened. The dumb network won. Perhaps the smart network can be pulled off successfully, but it won't be by humans, and I don't see anybody else around here trying.

The supposed "chilling effects" don't work out in practice. You will sometimes see people saying oh, China blocks TLS 1.3. Nope, many of the services you use now have TLS 1.3 and yet still work fine in China. "They just push people to a fallback". Nope, Downgrade Protection means that's exactly the same as just dropping all TLS connections.

TLS 1.3 doesn't change the correct way to interpose, it just makes yet more of the bad ideas that shouldn't work actually not work. If you were doing any of those and they broke, that's not TLS 1.3, that's because you were doing something that we already warned you shouldn't work. Stop doing things that can't work.

And it's true, such people keep demanding "transparency". I think they'd probably avoid some of their problems by just admitting that what they want is at best eavesdropping and that this isn't what other people think "transparency" means. There was a whole thread about this on the TLS working group list earlier in May. We get it, you would like to eavesdrop on people, but you don't like to admit this and you'd prefer if there was some way to pretend everything is fine, while, on the other hand, actually eavesdropping.

What's funniest is that the big players in this game want to play both sides so badly they can't even stop contradicting themselves long enough to write a document. The last version of draft-camwinget-opsec-ns-impact that I reviewed has such an obvious contradiction which I called out, it wants everybody else to use RSA key exchange, so that it can snoop all their TLS traffic, but of course implementers of the draft must forbid RSA key exchange, so that nobody snoops them.

> The dumb network won.

If the network was actually dumb perhaps we wouldn't have as many issues with routers fiddling with TCP as the GP described.

If was really, really dumb perhaps we could have managed to deploy SCTP and DCCP.

I wonder how much NAT interferes with new protocols, and if 'simple' stateful inspection with IPv6 would (have) made thing easier.

Nah, middleboxes are garbage. The security features they claim to offer are all snakeoil checkbox-ware, businesses and government agencies/state-controlled telcos use them because they have a nominal compliance obligation to do a thing so they buy a middlebox that they can wave their arms at when the auditor/commissar comes around.

Generalized "optimizer" middleboxes don't actually work in real world performance testing, with the possible exception of machines that apply dumb global queue discipline to fix a defective buffering device somewhere else in the network, but these don't need to do packet inspection at all, just byte-counting/early-drop.

Traffic shaping mostly only works in the endpoint, but aside from that, all you need is src/dest/byte count to do as good as traffic shaping as it is possible to do without being circumvented trivially (by the far end, without the users knowledge, thanks to the web letting the server script the client)

If you want to run a totalitarian state, do what actually competent totalitarian states do and take over the endpoints.

I'll give you that a lot of the middlebox problems are because they are garbage, however...

Dismissing the security features they are ostensibly delivering snakeoil is more than a bit unfair. There is a legit security context where absolute transparency & auditability is the right design objective, rather than absolute privacy. I'd argue the greatest failing we've seen with the evolution of TLS was the failure to recognize and address that reality.

For TLS if you build a middlebox which obeys the protocol invariants (e.g. see RFC 8446 section 9.3) then you're golden. This isn't so hard, it's almost embarrassing that anybody needed these writing down, but evidently they did.

This is true for QUIC too. Obey the invariants. You don't need to read all these complicated documents, just the invariants.

Now of course there are obstacles. First doing this is expensive. Doing things that don't obey the invariants (many of which also destroy your security) was cheaper. Too bad. If your supposed "legit security context" is so valuable you'll afford this.

Second doing this in effect requires confessing to the people you were "legitimately" giving "transparency & auditability" that you are snooping on them. Too bad, if your supposed "legit security context" really was what you pretend this conversation will be easy for you.

Yes, yes, you must obey the invariants. The trick is in the consideration of the invariants.

The problem isn't the cost or the "confession", because those are already part of the context. The problem is that in the process it creates new security risks that needn't be there but for the choices made by TLS.

In particular, the lack of separation of concerns between protections from eavesdropping, tampering and forgery proves to be quite problematic for environments where forgery & tampering are a serious concern, but eavesdropping is the nature of the context. It's why you see proposals for ugly hacks of TLS like eTLS; it's doing more harm than good.

Sure, in terms of raw features it looks like being able to uncouple the integrity verification would enable a client to choose to say "You can see what I'm doing, but you can't tamper with it". There are researchers who would like to do that, and last I looked the TLS working group invited them to turn that into an actual draft.

But it's essential in practice to remember that overwhelmingly the environment people are interested in is the Web, and on the Web those aren't actually different things again, in practice.

For example, suppose you're OK with me eavesdropping your Amazon purchases. You're buying a Lego robot for your daughter's birthday, you work for the investment bank, it's nice that we even let you do that from work and clearly we can't just let you do whatever you want with no oversight or you might rob us blind. We have a hypothetical TLS 1.4 which lets you opt in to this eavesdropping but nothing else, and you've done exactly this.

We eavesdrop your purchase, we were prevented from tampering with it. But, this is the Web. The eavesdropped TLS session contains a secret cookie value, which identifies you to Amazon. We can use this to impersonate you and do whatever we want, just as if we were able to tamper with your TLS session. The apparently more limited permissions are a mirage.

> But it's essential in practice to remember that overwhelmingly the environment people are interested in is the Web, and on the Web those aren't actually different things again, in practice.

If the problem is the web, then the problem should be addressed by the web, not TLS.

...and of course, part of the problem is that we've conflated nearly everything to the "web", so thinking that the web is somehow removed from this problem is a mistake.

> We eavesdrop your purchase, we were prevented from tampering with it. But, this is the Web. The eavesdropped TLS session contains a secret cookie value, which identifies you to Amazon. We can use this to impersonate you and do whatever we want, just as if we were able to tamper with your TLS session. The apparently more limited permissions are a mirage.

Yup, that's a legit problem, though I'd argue it actually further reinforces my point. This is a good example of a problem that stems from conflating multiple security concerns, rather than addressing them as separate concerns.

If there's an acknowledgement that your authenticated & tamper proof session is fully exposed to eaves dropping, then you start using the authentication mechanism itself to identify people or you develop an additional identification mechanism that is resilient to replay attacks (both of which are entirely possible) or people just inherently become aware of the reality that buying stuff on Amazon from an eavesdropping environment means that they will be impersonated on Amazon. What you don't do is think a "secret cookie value" is a secret when it so clearly is not.

The reality is that cellular data, satellite data, rogue WiFi, and fixed wireless all exist.

Transparency and Auditability are real, but they absolutely require endpoint security.

If you don't control the endpoint you don't know if it has an LTE chip, and if you do control the endpoint you don't care.

DPI security middleboxes for the enterprise are always and without exception snakeoil and/or checkboxware.

> DPI security middleboxes for the enterprise are always and without exception snakeoil and/or checkboxware.

If you don't care that your broker may be selling you down the river, or whether information is being leaked to an intelligence operative, that's a fair assessment. Unfortunately, much as we might wish otherwise, there can be conflict between privacy solutions and security solutions. The trick is to find a way to service both without compromising either.

If it's legitimate security it should've done via an explicit proxy not an attempt to break into every conversation and hope it works right 5 years from now.

Yeah, it's that kind of lack of understanding & consideration of the context that got us in to this mess.

I deploy these kinds of network security solutions to large enterprises as part of my job (typically healthcare customers, from a network MSP/MSSP VAR perspective these days) and as much as I like being paid to continually "fix" broken implicit systems it's not the way to go and never should have been. Either you control the systems or you don't, if you don't fully control you're going to fail at reliably breaking into the conversations to a meaningful level and you probably shouldn't have such critical information on systems you don't control anyways. That or you're trying to control/inspect someone else's system which is exactly what these things are trying to prevent.

> Yet we know a transparent, grokable, mungeable, backwards-and-forwards-compatible protocol that plays well with middle boxes is doable: HTTP

Not all traffic is web traffic, maybe not even most of it. There are so many things that would have absolutely wretched performance if you tried to implement them with HTTP instead of TCP. HTTP is way too generic to be truly performant.

I don't know that I'd hold up HTTP as an example of the way forward. ;-)

That said, the TLS v1.3 thing has been interesting. I tend to look at it more as a case of projecting one security context onto everyone else's security context, and the consequent problems involved. Security is tough like that.

> The “ossification” that QUIC deals with is primarily about internet routers who decide to dig into their IP packets.

Being 'opaque' to the network is not a foolproof choice, since it also makes stuff like NAT, congestion control and QoS a lot less robust. By exposing connection identifiers to the network, SCTP gains a lot of flexibility wrt. such scenarios compared to QUIC.

> it also makes stuff like NAT, congestion control and QoS a lot less robust

That's a feature, not a bug. NAT is cancer. It killed the open internet by enabling a particular lazy form of "security" that forced everyone to hand control of their most important data to cloud services.

QoS is worse but has less fallout due to less deployment.

>NAT is cancer. It killed the open internet by enabling a particular lazy form of "security" that forced everyone to hand control of their most important data to cloud services.

Sorry I dont follow. What Cloud Services?

All cloud services.

Personal computers and devices are not allowed to talk to each other on the internet. They are only allowed to talk to specially designated servers -- specially designated by virtue of having a public IPv4 and open port. Enforcement happens through liability: if you put data in a cloud service and the cloud service gets hacked, it's the cloud service's fault, but if you run a program on an open port and the program gets hacked, it's your fault, not the program's fault. This is an arbitrary social choice that follows directly from enshrining NAT as a "best practice." Even the term "open port" presupposes that there is something unnatural about plebian computers talking to each other on the open internet! This effectively forces services to be centralized, or at least pushes very strongly in that direction.

Of course, we can quibble: what about video conferencing? What about gaming? They use direct connections! Yes, but they pay an extraordinarily steep complexity and reliability price to obtain them and the solutions still wind up using centralized servers. They are exceptions that prove the rule because they allow us to observe how "stepping out of line" is discouraged. It's an accidental line drawn by monkies rather than an intentional line drawn by lizard people, but it's strongly enforced and society-shaping all the same.

It's totally wild to consider the social and market consequences of NAT, the unassuming lazy and kludgy security hack. It got out of hand and reshaped the internet, the technology market, and society from the ground up! That's hardly even an exaggeration!

NAT is literally the only thing stopping every single personal device in the world from being hacked. It may be a kludge, but it's the best kludge in all of technology. And what do you have against QoS? Would it be better if your VoIP calls stuttered every time you loaded a fat web page?

> NAT is literally the only thing stopping every single personal device in the world from being hacked.

This is so wrong that it must be satire. NAT provides the very most basic protection possible, easily replicated (and surpassed) by every firewall that comes stock with every modern OS, and is completely insufficient (and not even necessary) to prevent personal devices being hacked.

I'll buy this for firewalls in end-user routers, but if you put the firewall on the personal device it's gonna get disabled the first time it supposedly interferes with a videogame or a movie stream or whatever. :(

NAT enables dogshit endpoint security, yes, but that's the problem. Without NAT as a crutch, endpoint firewalls and service security wouldn't suck. Compared to other feats of security engineering the industry has undertaken in order to make centralized cloud services happen, namely JavaScript and the ability to safely run untrusted code from the nastiest corners of the internet at high performance, this would be downright trivial. It just never happened because we had NAT.

It's really not. Like, really, really not. I won't try to make the case here, but just know that assertions like that aren't going to come across as credible. The systemic effects from NAT have actually made things worse.

No it isn't, that's basic stateful firewalling. NAT in no way provides security. At all.

The fact that a kid with a port scanner can't remotely connect to and exploit your printer is due to NAT not being able to route the traffic, whether you have a stateful firewall or not. The majority of internet-addressable devices in the world do not have firewalls. Hence, NAT keeps most devices safe from drive-by RCE. There are plenty of attacks to get around that, but by default, nothing else protects random devices like NAT does.

> The majority of internet-addressable devices in the world do not have firewalls. Hence, NAT keeps most devices safe from drive-by RCE.

According to System Preferences, my Mac's firewall is currently turned off. Here are the current non-NATed IPv6 addresses on it:

   $ ifconfig en1 | grep inet6 | grep -v fe80
 inet6 2607:f2c0:93c7:fe00:144f:b9a3:bcd4:ef6 prefixlen 64 autoconf secured 
 inet6 2607:f2c0:93c7:fe00:e04c:f356:f2:b9e2 prefixlen 64 autoconf temporary 

You should be able to ping(6). Can you connect to tcp/22? If not, it shows that my router's stateful packet inspection (SPI) works even when NAT isn't present.

* http://www.ipv6scanner.com/cgi-bin/main.py

The results of an nmap of one of those address from the system itself:

    PORT     STATE SERVICE REASON
    22/tcp   open  ssh     syn-ack
    111/tcp  open  rpcbind syn-ack
    1022/tcp open  exp2    syn-ack
    2049/tcp open  nfs     syn-ack

If you have a device doing NAT, then that device is already doing SPI, and so the NAT part is redundant. It is the SPI that gives you the security.

Well first, IPv6 doesn't need NAT, and mostly never uses NAT. If IPv6 devices used NAT, they would be protected. But since they aren't using NAT, they aren't protected. You're right that devices that enable IPv6 (and network routers that give out IPv6 addresses) are not protected by NAT, but luckily it's still a small amount of traffic/devices worldwide. But this is changing, and so NAT won't help people for much longer.

Second, SPI isn't used for security here. It's true that SPI is used by NAT, and that SPI can be used to prevent, say, a specially crafted packet from passing from a public network through a router into a private network. But something other than SPI has to actually enforce that. That "something" is usually extra features of the network stack, like disabling source-routing, reverse-path filtering, ARP filtering, etc. All SPI is doing is helping the NAT engine track the connections it is translating.

For "good faith" traffic (that is, not crafted by an attacker to work around crappy NAT routers) NAT still provides security. Actually it's not NAT at all that's providing the security - it's just the routing. You can't pass traffic from a public network into a private network, and almost all devices behind NAT are on private networks. So NAT provides security merely by keeping devices on a private network.

> But since they aren't using NAT, they aren't protected.

I just demonstrated that protection exists without NAT.

> […] but luckily it's still a small amount of traffic/devices worldwide.

I don't think this is correct. Just about all devices on a cellular/mobile network nowadays are probably using IPv6 natively, with CGNAT for IPv4. That's a lot of devices.

* https://blogs.akamai.com/2016/06/preparing-for-ipv6-only-mob...

An experiment: temporarily disable Wifi on your cell phone, go to your browser, search of "what is my ip address". Chances are that you'll see an IPv6 address there: if using Google they'll return something an IPv6 address at the top, and a list of different web sites that do the same thing. If you go to a website you'll probably get an IPv6 and an IPv4 address.

> But something other than SPI has to actually enforce that.

Connection tracking is sufficient for most modern protocols (i.e., not (active) FTP):

> A stateful firewall keeps track of the state of network connections, such as TCP streams, UDP datagrams, and ICMP messages, and can apply labels such as LISTEN, ESTABLISHED, or CLOSING.[2] State table entries are created for TCP streams or UDP datagrams that are allowed to communicate through the firewall in accordance with the configured security policy. Once in the table, all RELATED packets of a stored session are streamlined allowed, taking less CPU cycles than standard inspection. Related packets are also permitted to return through the firewall even if no rule is configured to allow communications from that host.

* https://en.wikipedia.org/wiki/Stateful_firewall

* https://en.wikipedia.org/wiki/Firewall_(computing)#Connectio...

> Actually it's not NAT at all that's providing the security - it's just the routing.

Yes, that's exactly my point. And as part of routing packets one can inspect their headers and drop or allow them as desired.

I think we're sort of agreeing on the same things. NAT protects devices, and stateful firewalls protect devices, and if you have neither, you're not protected. My point was that more devices exist behind NAT than come bundled with a stateful firewall, hence NAT protects more devices. The NAT devices/routers might additionally have a stateful firewall (which really only protects the router, not the other devices), but I wouldn't trust it as much as the security of a non-routable network.

> My point was that more devices exist behind NAT than come bundled with a stateful firewall, hence NAT protects more devices.

The popularity of a mechanism does not necessarily correlate with the effectiveness of its security. The fact that we're using IPv4 with NAT+SPI versus IPv6 with SPI is simply an accident of history.

If IPv4 had been designed with 64-bit addresses, or even 48-bit ones (like Ethernet MACs, which we still haven't run out of), then NAT probably would not have been invented, and we'd be using 'simple' (SPI) firewalls (see Cheswick 1994).

> […] but I wouldn't trust it as much as the security of a non-routable network.

Non-routable networks are not more secure than routable ones with SPI; at best the two are equal IMHO. Non-routable networks may actually be worse because of a false sense of security: all it may take is one end-point compromise and the enemy is one the other side of the moat.

Yes I agree with this but do note that in your gp you stated "every single personal device", not "half of all personal devices"

Even stateful firewalls don’t protect you from being hacked. Hacks occur when you run untrusted code on your machine. That untrusted code can easily bypass your stateful firewall by initiating a connection to a C&C server.

Thankfully, this is going away now with IPv6.

Unfortunately not. Most consumer and SOHO routers have reflexive ACLs enabled by default, and your average Joe doesn’t know or care to turn it off.

Oh, interesting, what is it exactly ?

Surprising that with over a year of discussions about this subject I only first hear of it ?

Would this be the reason why at least tens of millions of consumer routers don't have IPv6 firewalls (technically, opt-in firewalls, but you know how that goes...), and don't have their networks hacked ?

Wait, does it even work with UDP, which doesn't have a concept of "connection" ? (Especially IPv6 (no NAT) UDP ?)

I want to believe!

Well, IPv6 seems to be today (both in popularity and speed of adoption) where UTF-8 was about a decade ago. So, hopefully in a decade..?

NAT is a hack that only ever existed due to IP address exhaustion, and I’ve yet to find a good reason why congestion control and QoS handling in a router needs anything beyond what is already in the IP headers. SCTP’s flexibility is irrelevant since nobody uses it, arguably because of how much the Internet has ossified around TCP and UDP.

Hey, that's not true. People use it... in the telecom industry and everyone using WebRTC's DataChannel (SCTP over DTLS actually).

The protocol is probably mostly fine, but there are no quality library for using it in a general purpose way. Because of that, no one can really use the protocol to its full designed capabilities anyway (and the ossified network nodes don't help either).

It doesn't matter if SCTP has good features if no one is willing to implement quality libraries for it. The protocol exists for a while and only one general purpose library exists for it that's widely used. And it's not without a lot of flaws and limitations.

There are good reasons no one wants to invest in it, and I'm sure they did consider it.

I am sure there are even more implementations that I am not aware of.

* https://github.com/pion/sctp

* https://github.com/aiortc/aiortc/blob/main/src/aiortc/rtcsct...

* https://source.chromium.org/chromium/chromium/src/+/main:thi...

* https://github.com/sctplab/usrsctp

People don't make these decisions for technical reasons only. Career wise it is a bad choice to spend your time working on pre-existing technologies. You don't become a distinguished engineer by iterating on existing technologies. You become one by being the creator of something new.

I think QUIC is great and does a good job solving the problems it was designed to solve. It is disingenuous to pretend these decisions were made only for technical reasons.

3 of those are related to DataChannels in WebRTC, which at this point is baked into the standard and can't be replaced by another protocol. The SCTP in DataChannels is quite limited, that subset is reasonable to implement somehow.

Only the last one is really general purpose and has had a lot of security issues as well.

Fun fact: I do work on one of those implementations and with another one, and it's out of necessity rather than a career choice.

I know you understand libusrsctp Florent. I created Pion, I feel prety comfortable with SCTP and WebRTC in general as well.

RtcQuicTransport tried to replace SCTP and it didn't work. It would be possible to replace SCTP if something better was available. SCTP has lots of great stuff like FORWARD-TSN. QUIC didn't offer anything compelling over it. Also would come with a huge cost of making WebRTC larger and losing interop with all the existing clients.

libusrsctp has had security issues, but I don't think the protocol is the problem. The issue is C/C++. QUIC implementations are going to have the same class of bugs. Chrome/libwebrtc has plenty of security issues in other areas besides SCTP. Rust/Go doesn't fix everything, but one less thing to worry about at least.

If Google has a chance to use an industry standard or use their clout to create a new standard, you get one guess at what they're going to do.

Having a standard doesn't mean you're not allowed to innovate and create another one.

SCTP dates from RFC2960, first draft was published in 1999. It had enough time to get traction, and it didn't. Why would anyone build something new with it now, knowing that it doesn't answer some of the issues with the current Internet?

Even WebRTC's DataChannels were introduced at the IETF in 2012, that's 9 years ago (although ironically, the RFC just got published in January...).

Correct me if I'm wrong, but bypassing NAT, congestion control and QoS is also a benefit when you want to build things like P2P services where you want a direct connection between peers and full control over the connection itself. QUIC will help a lot with those use cases.

I'm happy whenever anyone brings up SCTP. Such a maligned protocol. Congestion control and QoS, however, are a loser's game. If you're running into realistic issues meeting demand due to statistical multiplexing breaking down to the point you must rely on caring about the internal state of traffic streams, you should have thrown more link hardware at the problem long ago. Many dumb pipes > fewer smarter pipes.

>NAT

Oh, didn't you know, IPv6 was going to solve everything. No more NAT. We're just going to assign everyone an IP at birth so people can continue to ignore the User-Agent/User distinction.

QoS has its uses, and you don't have to look any deeper than TCP/UDP headers for it to be good enough. It allows VOIP/IPTV services to continue working under DDoS conditions for example (ISP level).

IP TOS flag and TCI field in Ethernet header (802.1Q header) should be sufficient, without looking at TCP/UDP/SCTP headers.

NAT does not need anything more than port numbers or UDP cannot be used with it.

As for QoS going beyond port number, source IP and rate of packets asks for troubles.

I wonder : IPs are considered PI (heh). So does this mean that badly implemented QoS can end up violating RGPD ?

Are deep packet inspection and modification boxes really that prevalent outside of the Great Firewall of China and its imitators (Russia has something similar now)?

If QUIC gets in the way of these countries they will simply set up a decrypt-and-recrypt TLS proxy and block connections that don't honor its certificate. Lots of banks and trading firms already do this to their employees for regulatory purposes; there is no technical hurdle.

> there is no technical hurdle

As someone who has been forced to use ZScaler, there is a slew of technical challenges. 80% of the traffic can be handled "easily" with a custom CA cert, and the rest have varying issues ranging from client problems to server problems to problems in the proxy itself. Just trying to clone a GitHub repo over HTTPS breaks with ZScaler.

I can't comment on this specific product, but if the transparent proxy is causing you so much grief why not just explicitly configure your machine to use the HTTPS proxy?

HTTPS proxies are not voodoo. There are plenty that work quite well.

Actually there is "voodoo" in both transparent and non-transparent proxies, but the bigger problem is that ZScaler's entire raison d'etre is to be a transparent proxy using Carrier-grade NAT. But the point is, ZScaler is a widely-deployed commercial solution, and it does indeed have technical hurdles. Whether it should theoretically work or not, I'm telling you in practice there are problems, and this shouldn't be dismissed out of hand as an outlier.

You really seem to have a beef with this one particular commercial product that I don't have access to.

I've used plenty of other HTTPS proxies and the well-written ones all worked fine. Many are open source. I think you need to direct these comments to whoever is charging you money for this "Zscaler" thing.

But isn’t the “damage” these middleware layers do actually good optimizations/features that people pay for? The ossification is the problem, not the ability to optimize IMHO.