It does not need HTTPS to be secure, but it would need HTTPS to add privacy, for which the protocol has none at the current time.

As explained on the website, HTTPS would not add meaningful privacy at all, because without significant other changes the architecture, what you're doing is still downloading files from a very limited set. The size of the files in most cases is unique so that an onlloker can tell what you downloaded, encrypted or not.

I find this argument not very convincing. Suppose an attacker wants to track people downloading stuff over APT. This is what they would need to do:

In case of HTTP - Step 1: Read the HTTP request payload. Step 2: There is no step 2.

In case of HTTPS - Step 1: Build an index of all possible packages and their sizes. Step 2: Reassemble HTTPS response traffic into individual HTTP responses. Step 3: Look up the response length to the corresponding package. Step 4: In case of identical file sizes, make some sort of model to find out which packet it looks to be based on other packages downloaded (?).

Yes, it's still possible to track people's packages all the same. But you have to have a have way more determined and prepared attacker - it cannot be as easily be done through casual eavesdropping. It's a false equivocation to say it would not add meaningful privacy, as your attacker model changes from casual eavesdroppers to more determined attackers.

You might not care for that particular distinction, and I agree people should have the choice to use HTTP or FTP for APT when selecting mirrors. Unencrypted APT is plenty secure, but encrypted APT is really a little better. In my opinion, there should not be so much resistance for HTTPS in default configurations (e.g. the Debian project could easily require this for their official mirrors around the world). Let's Encrypt makes this so easy, there's no argument anymore in my opinion.

That's because you are glossing over a lot of packet inspection deeper than the TCP level with a glib "Read the HTTP request".

You might want to think about existing surveillance systems. Analysis of telephone traffic is often done purely on the CDR (the caller, callee, and length of call, in simple terms) without the equivalent of deep packet inspection to read the HTTP request, which would be analysis of the actual audio data themselves. The HTTPS case would likewise need just the total octets transferred over the TCP connection for fingerprinting.

There's a lot of glib handwaving in this discussion about identical sizes, not based upon actual measurements of the Debian archive. I quickly looked at the package cache in one of my Debian machines:

    jdebp% ls -l|awk 'x[$5]++'
    -rw-r--r-- 1 root root      3314 Feb 16  2018 nosh-run-freedesktop-system-bus_1.37_amd64.deb
    -rw-r--r-- 1 root root     35190 Dec 14  2016 redo_1.3_amd64.deb
    -rw-r--r-- 1 root root   1114546 Feb 25  2018 udev_232-25+deb9u2_amd64.deb
    jdebp %

It turns out that in practice size alone almost does uniquely identify package in this sample. The other file that is 35190 bytes is version 1.2 of the same package, leaving just 2 possible ambiguities out of 847 packages. It seems likely that this holds after encryption as well.

So the remaining question is how much HTTP pipelining ameliorates this, which no-one here has yet actually analysed.

I'm not sure I understand your reply. Are you replying to me as if I said that determined attackers cannot trace back HTTPS traffic to individual APT packages? Because I said no such thing.

I just made the distinction between casual eavesdroppers and determined attackers. Those determined attackers exist and are quite capable, I'm sure. I said as much in my post.

You might also want to look into your use of the word 'glib' here. I find it an uncharitable interpretation of my post to call it 'glib' or 'glib handwaving', to be honest. Makes it seem to me as if I should be defending something I said, but I'm not sure what.

There's really no difference between a "casual" eavesdropper and a "serious" one. In what world do you live in where the former camp even exists? No one is casually spying on your apt updates, and anyone who is "seriously" spying on your apt updates can trivially manage to identify them by size. HTTPs really doesn't add anything here.

Of course no-one is spying casually on HTTP APT traffic specifically. Nobody is arguing that strawman - nobody here is "living in that world", give me some credit please.

But people spying casually on HTTP traffic in general do exist. People able to spy on HTTP traffic in general casually is one of the main reasons we care about HTTPS in the first place. Even though people can do a targeted content length analysis for nearly all other the stuff we read/watch/download online, too. We still care about HTTPS for all of that. And we should probably care for that with APT too, if only a little bit.

TL;DR HTTPS gives you potentially more confidentiality but not guaranteed as known vulnerability exists which an advanced attacker can exploit. You should not assume confidentiality when using APT over HTTPS. The severity of this issue in a CVSS is going to be very low because it is only an information leak.

What?

ISPs, for example, eavesdrop us all the time, and they do it quite casually. They will modify your unprotected HTTP requests, inject ads, log everything they are able to, and sell the data if they can.

Wouldn't help. If almost all of the files' sizes are unique (I'm pretty sure compressed package sizes aren't even block-aligned), and you know the sizes ahead of time, and you can make test requests for samples of what headers are being sent/received, it's trivial to calculate which combination of packages would result in a given stream length using pipelining. You'd have to add countermeasures like padding or fake data.

You could quantize these with up to 10% padding and cause a very large number of collisions, but that wouldn't be useful without HTTPS. Is the core argument that privacy is not attainable, or that it is not valuable?

I think the core argument first of all is that it is not attainable trivially by "just using HTTPS". So it's a question of costs vs. benefits, where the costs are pretty big (change the whole infrastructure).

This is wrong. A great deal more privacy is attainable by trivially using HTTPS. Privacy in the presence of stream inspection is more difficult, but attainable by padding files to have quantized lengths.

You are entirely incorrect in this instance, for all the reasons laid out in this thread.

Looking at stream sizes is not "advanced deep packet inspection", it's in fact the opposite of that.

Up to 10% padding likely isn't sufficient to provide privacy. Think of the long tail of large packages.

Quantizing up to 10% collapses 43,000 values into 120. It's not perfect but it removes a huge amount of information.

How do you arrive at 120? I guess that makes sense if the biggest packages is within 92709x (1.1^120) the size of the smallest. But that doesn't seem like enough range, just eyeballing it. If you have a 1kB package at the low end, I'd be surprised if Debian didn't have a package bigger than 92 MB.

Presenting it as 120 from 43,000 is a bit of an oversimplification, because the average isn't meaningful. The long tail is going to have the worst privacy and the small packages will (probably) have the most.

A scheme like this might be workable but requires being really careful about the security properties you're claiming (i.e., of those 120, probably half are unique, large packages). And obviously, this scheme requires up to 10% additional bandwidth, in the case of the chosen 10% threshold. If buckets change over time, packages moving between buckets may leak a lot of information.

"Step 1: ... etc."

This sounds less like some sort of massively impossible barrier to overcome and more like a Project Euler problem, and one not all that far into the sequence, either.

One of the things you have to overcome if you want to think like a security person is that, yes, there are attackers that will put some effort into attacking you if you are a target of any consequence, certainly effort far exceeding what you just described. I've watched some people at the company I work for have to overcome that handicap myself. Yes, there are attackers that are not just script kiddies and actually, like, have skills and such.

Attackers won't jump through infinite hoops, but getting a foothold on a network somewhere where they'd like more access, seeing that they can watch a new system in your network getting provisioned, and cross-checking that against a list of known vulnerabilities by looking at package sizes would be boringly mundane for them, not something wildly exotic.

There are two reasons you want to compromise a host:

1) You're building a botnet (or, these days, are crypto mining). In that case you're not targeting a specific machine, you just want many of them.

2) You want to exfiltrate information from a host, or sabotage it. In that case you're targeting a specific machine.

I'd argue that in both cases, the proposed attack vector of inferring installed software versions through apt downloads is inferior, or at least more involved. In case 1) you're better off scanning for known vulnerabilities or make use of shodan and the likes. In case 2) you're probably going to probe the server anyways. It might take a little more time than if you just had a complete list of installed packages and their version (given you were somehow able to eavesdrop on the host in the first place), but you'll most likely determine at least what OS is running and what technology stack their internet-facing services are running on after some nmapping.

Or, looking at it from the other perspective, I wouldn't really feel much safer if apt were using https. I'm not against it, but I don't think it's a priority, especially if it needs a lot of coordination between different people, which always turns out to be very time consuming. Just being fast with updating packages seems a better investment of that time.

> Or, looking at it from the other perspective, I wouldn't really feel much safer if apt were using https. I'm not against it, but I don't think it's a priority, especially if it needs a lot of coordination between different people, which always turns out to be very time consuming. Just being fast with updating packages seems a better investment of that time.

This is exactly my position, to be fair. We're all bike-shedding here as far as I'm concerned - including this very website. I think the position that HTTPS doesn't help you is a little bit disingenuous, and the only fair position is that "coordinating this stuff takes time and effort we don't feel is worth the negligible advantages" (as you say, and as this website says) is a more acceptable argument than "the negligible advantages don't exist" (as this website seems to also want to say).

A: Assuming package privacy could somehow be protected (a question I leave to part B), then I agree that there is no improvement in security against capable attackers: they would have to focus on hacking the APT servers, on which the attack surface can be minimized, and monitoring and logging any deviations can be tracked and published for all to inspect.

B: if the packages are concatenated to each other and a random length noise string, we can substantially frustrate nation state / ISP level attackers to the point of forcing them to get this information from the endpoints themselves: Either end user, or the APT server must already have been compromised.

1) End user not yet compromised: in order to capture these, they must attack the APT server.

2) End user already compromised: on each update of all compromised users, information is sent to C&C, so this would produce lots of opportunity for attentive users to discover the implant.

When focussing on the APT, which would give the cleanest record of attack surfaces, the community can put man power on designing minimalist APT servers, and inspecting published deviations in communications can lead to uncovering 0days.

EDIT: changed disagree to agree, as I (incorrectly) thought you were arguing it would not make attacks more expensive, woops!

Any self signed certificate? Then the attacker who is already capable of intercepting packets can just serve whatever and proxy your request. Your argument to the main argument falls short just the same..

Agreed, went a bit overboard there. Removed the part about self-signed certificates.

Steps 1-4 are very easy; they just require some dev work. Thinking an attacker won't bother because it sounds annoying to implement is security by obscurity.

No, it's security against a specific class of attacker that your threat model is considering.

We know there are nation states that build profiles of each user based on their HTTP requests, but we don't know of any that have written custom software specifically to target Debian users.

It would take them half an hour to write that custom software. Under what threat model is "nation-states that can't spend half an hour writing code" a meaningful adversary? Note in particular that they can identify users by previous traffic statistics (and just normal traffic flow statistics that might be kept as a routine log by any network administrator, even); they don't need to write the code in advance. Given the number of bytes transferred, the times, and an archive of the state of the Debian archive (which is publicly available), they can always identify past downloads.

(Would it help if I wrote that code right now and put it on GitHub?)

> It would take them half an hour to write that custom software.

It would take a software engineer half an hour to write that custom software. It would take a government years to amass the political will to target such a small section of the population, and then potentially hundreds of thousands of dollars for a government contractor to offer a solution and implement it.

There's always going to be a big difference in threat level between a piece of software which already exists and a piece of software that could exist. For example, when you're snatched off the streets by the secret police, and they go to investigate what you've been doing in the country, they might be able to request from HQ a list of HTTP addresses fetched from the IP address associated with your apartment, but they're unlikely to be able to request that HQ write some software to go back retrospectively and count bytes of individual connections you made.

> (Would it help if I wrote that code right now and put it on GitHub?)

No, but it would help if you wrote a patch for APT which made it use HTTP range requests to hide the size of the files it downloads. That should only take half an hour, right?

I continue to be confused by this threat model where all government agencies you're worried about are plagued by massive levels of US-style governmental bureaucracy and can't get anything done, yet they're capable of being meaningful threats. (Also where the only entities you're worried about are government entities.)

The entities I'm worried about have bought off-the-shelf surveillance tools which record the HTTP requests associated with each IP/MAC address. This is a minimum viable product for governments and ISPs (not to mention businesses, like hotels and coffee shops), and it is reasonable to think that such a system is deployed on orders of magnitude more networks than a system that tries to infer Debian package downloads from counting bytes of HTTPS traffic.

> Read the HTTP request payload

The thing to note here is that the only reason this seems easy is that there is tooling readily available for such a task. If you didn't have such tooling, you'd find it more difficult to implement then your HTTPS case even _with_ tooling.

The same principle applies to your HTTPS case. Your argument disappears as soon as there is tooling. That tooling only needs to be written once. Perhaps it already has been done and exists in the circles where people want to surveil apt users. One possible reason such tooling isn't widely available is that apt doesn't use HTTPS by default, and one outcome may be that if apt switched to HTTPS the tooling would appear.

I have half a mind to write the tooling and publish it just to eliminate this argument. It really isn't very difficult.

There's a big difference, privacy-wise, between being able to say "X is talking to this debian mirror and thus probably running debian" and "X is downloading exactly these packages from this debian mirror".

The point being made is that the fact that X just talked Y bytes to this Debian mirror is enough to know exactly which packages were downloaded.

The argument that https obfuscates which packages you download is not a good one, and may cause users to unnecessarily worry about the implications (and conversely, that they end up "more safe" if that was not the case). If that type of privacy is desirable you should probably use something like Tor.

When I say `apt-get install foo` and it brings in 47 other packages, the problem gets exponentially more difficult. "He downloaded 1,432,509,104 bytes; what packages were those?" is more or less O(2^n): https://en.wikipedia.org/wiki/Knapsack_problem

If you download exactly one package, it may be easy to deduce which one it was (assuming that the protocol overhead is identical each time, and that changing timestamps and nonces doesn't affect the byte length whatsoever, etc.). If you download more than one at a time, which is common with Debian, then the problem is a whole lot harder.

An HTTPS stream can be side-channel (ie size, time) broken down into black-box HTTP/1 requests quite easily. Remember, even with Connection: Keep-alive, you still have to request every file synchronously after you're done with the previous one.

> Remember, even with Connection: Keep-alive, you still have to request every file synchronously after you're done with the previous one.

Wait, why? I don't know of any reason HTTP clients can't pipeline requests.

They can but they do not because servers are broken (and terribly so) and it brings less than zero gain for big files.

See, debian does not want to manage specific mirror server features if they don't have to. If they were in that position they'd make their own protocol.

Right, that makes sense. So it's not so much a client limitation as a minimum common denominator for their hosting infrastructure.

its not because a naive implementation of privacy enhanced APT would fail that all implementations would failt at protecting privacy. Concatenating all the packages and a variable length noise string together, should go a long way.

It would be more fruitful to discuss the different ways an attacker might deduce what software was installed from a naive implementation: download sizes, download date (i.e. new update available for package P, then a substantial fraction of users who were downloading from the server that day were probably installing P etc...

In theory an onion router might substantially improve the situation if the attacker has a hard time identifying which server the user is talking to, and thus making it hard to identify if the user is even installing anything at all...

sadly I don't trust TOR as long as I can't exclude a specific attack scenario I have always suspected about TOR but never actually known to be present...

> its not because a naive implementation of privacy enhanced APT would fail that all implementations would failt at protecting privacy. Concatenating all the packages and a variable length noise string together, should go a long way.

Sure, but then you are not talking about "just use HTTPS", you're talking about creating your own protocol and requiring all APT packet sources to speak that protocol, requiring a specialized server software, where currently they can just use whatever HTTP server they want. Switching the whole infrastructure and installed base over to that would be a massive multi-year project, not just a handful code and configuration changes.

Note I use the phrase "naive implementation" and was never talking about nor clamoring for "just use HTTPS".

The real discussion is not "blindly use HTTPS, or leave it like it is", for me the real interesting question is: can we design a package distribution system that preserves privacy against nation state level actors? can we virtually force those to attack the ATP servers themselves? could we use oblivious transfer ? could we design a fresh minimalist onion router (as opposed to bloated TOR) for package distribution?

> can we design a package distribution system that preserves privacy against nation state level actors?

That's a great question. Also, can we do it on top of the current Debian infrastructure (HTTP and everything)? Or do we need to change anything?

> Also, can we do it on top of the current Debian infrastructure (HTTP and everything)?

I'm pretty sure that is not possible, because the current infrastructure is just plain old HTTP file servers (anything that can sling bits will do) run by whoever fancies being a part of it.

If you create an onion layer like the GP proposes, I don't see any loss from it running over HTTP servers.

It already exists. It's called tor.

But the question here was why https isn't default in apt, not why tor isn't.

you're replying downstream of my comment containing "for me the real interesting question is:..." where I generalized the question away from the false dichotomy "keep apt as it is, or make https default in apt"

> The real discussion is not "blindly use HTTPS, or leave it like it is", for me the real interesting question is: can we design a package distribution system that preserves privacy against nation state level actors?

That is certainly an intersting theoretical question, but in practice there is also the question of the costs of something that requires you to change and complicate the whole distributed infrastructure vs. the benefits - are there actually real people who need privacy against "nation state level actors" specifically concerning the Linux packages they install?

what about journalists, whistleblowers, strictly vocal critics of the centralized surveillance state [0], lawyers, politicians, ... ?

[0] https://news.ycombinator.com/item?id=16947652

EDIT: just adding, we also don't know what the cost is of the most efficient privacy presevering distribution method actually is. only when people investigate and try will we find out.

> variable length noise Just thinking... could this be achieved by simply adding some extra response headers on the mirror side? That is the response could contain some headers like

   X-Unnecessary-Header-0: [random-string]
   X-Unnecessary-Header-1: [other-random-string]
   ...

This could be enough to insert random-length noise without the need to invent any new protocol. Of course this would only be effective for smaller packages as I assume that headers size is limited and as a result this would significantly change the perceived download size only for smaller packages.

that might work, and the larger packages might be sent as a sequence of parts, so that essentially all packages have the same size

other things to keep in mind is the dependency graph, some large packages might be rather unique in total filesize downloaded

The post you replied to outlined a way to determine which packages were downloaded for the HTTPS way, so I don't get what you're trying to say here.

Preventing script kiddies or dickhead ISPs logging your traffic is meaningful privacy.

That is, in fact, the main threat everyone should protect against.

Privacy against script kiddies and dickhead ISP is valuable. Not every scenario is a determined attacker targeting you personally.

This can easily be solved by inflating the wire size of the file but the received download, transmitting junk packets, that fail to decode, but still look like encrypted noise to an eavesdropper.

Not everyone uses fast unlimited internet.

Pretty much everyone using Debian flavored Linux is.

That doesn't sound like an argument against it to me. It just means that in addition to HTTPS, they also need to make "significant other changes the architecture".

...which would be a huge undertaking given the vast installed base and the fact that packet sources currently don't run any custom server software at all, which would need to change.

I'm not so sure about that. Assuming all the packages are downloaded over a single connection, you can easily pad the response client-side via HTTP range requests, for example. No special server software required; just a normal HTTP server.

You're downloading form a third-party volunteer. What protocol can provide privacy given that condition?

There is a huge difference between the third-party knowing what you're doing, and everyone in between knowing what you're doing.

The HTTPS everywhere movements are attempting to make privacy the default rather than the exception, but are of course done knowing that the server will always know whats up. The point is to make it so that only the two parties concerned, the server and the client, comprehend the communication rather than the entire world

Okay, suppose I want to know what packages you're installing and updating. I have two routes:

1. I gain access to a router near you.

2. I rent a sizable server at the hetzner/ovh/… location that's closest to you and volunteer to run a mirror for the OS you're using.

Both are somewhat uncertain (your traffic might flow via a different route, you might be load-balanced to a different mirror) but the uncertainty seems comparable. Option 2 seems so much easier that I have a real problem seeing the point of even attempting option 1 if all I want is the information option 2 would give. Perhaps someone can explain?

You'd have to take control over my mirror. Making a new mirror will not get you any traffic unless people actively choose to use it. Therefore, your option 1. is to control a part of the public route, and option 2. is to control the mirror of my choice.

However, any argument against using encryption for privacy for APT can equally be applied to any other traffic. Do you trust your public internet route enough to let your traffic run authenticated, but unencrypted? Chats, news, bank statements, software updates?

Even if content cannot be modified, it can still be blocked or made public. There are quite a few nosy governments that would like to know or block certain types of content, software packages included.

Lots of people use the nice hostnames like ftp.us.debian.org. It wouldn't be too hard to get included in that host name if you're determined. I haven't looked into the requirements, but I'm pretty sure it's a) be technically competent enough to run a mirror (it's not hard) b) have a lot of bandwidth. c) be organizationally competent enough to convince Debian that a and b will hold for a long time.

You will instantly spot blocking because apt reports connection failure and hash failure and replay attack on update list is ineffective.

As for privacy, eh? It is visible you're connecting to a debian mirror and what size of update list you're getting. Barring that, indexing packages by size is trivial.

You want true privacy, you'd have to use Tor or such.

Option 1 is hard for individuals but easy for state actors.

For example, they might want to know what versions you're running (by looking at what updates you _didn't_ download) so they can target you or lots of people at once.

Since it's known to which host you connect and the pattern of access is known to, it's a reasonable guess that it would be possible to infer the list of packages that you're downloading from observing the encrypted traffic.

Instead of JUST that third party knowing what (possibly vulnerable) packages you have, you are now letting everyone know what possibly vulnerable packages you have.

> you are now letting everyone know what possibly vulnerable packages you have.

Erm, what?

The number of people, who can listen to (much less — modify) your traffic is very small. It is basically your ISP (who is supposed to offers you services in good faith, not spy on you) and a number of engineers, who maintain Internet backbone. That's far from "everyone". Some SSL evangelists make it sound like everyone's traffic is permanently broadcasted to everyone else in the world, but it is not.

As for "vulnerable packages", the most certain sign, that someone does not install security updates, is lack of traffic between them and update servers. But that's orthogonal to use of encryption.

"everyone" in this context obviously means everyone on the path, and any attackers that have compromised nodes along the path. See the Belgacom hack by 5 eyes...

And I those attackers can still tell everything based on the target IP and the payload sizes.

there is no proof that this is true in general, so it is worth trying to find 1) an inefficient way in order to 2) postulate an efficient way...

for example, overlay onion routing, size blurring by appending random length random bits,... with oblivious transfer even the APT-server does not know what you downloaded (but that would require a large amount of information..., nevertheless oblivious transfer might still be a useful tool when used as a primitive, perhaps just to send a list of bootstrap addresses for p2p hosting of the signed files etc...)

Couldn’t this be used to identify possible hosts that are running exploited software. So you watch a target and keep track of their installed packages. You also monitor for zero day exploits. The instance you have identified a zero day you also have a list of high probability targets to test it out on or exploit. Privacy is more then just I know you have blue eyes or brown eyes

Genuine question: what are you doing with APT that requires privacy?

In the past cryptographic export was illegal in the us and many other countries, it can still get you put on a list.

In some countries messaging apps like signal and telegram are illegal.

There is no telling what seemingly benign software will be made illegal in the future for political reasons.

Privacy is always a requirement because of these reasons.

Size fingerprinting defeats such "privacy". If you need this, use Tor instead.

Cause pushing the barrier of how easy it is to detect is a bad thing.

This is very asymmetric though. The cost for an attacker is very low. The cost for Debian and it's mirror network is very high.

Given that the cost of implementation is high and the protection is minimal the decision to not do so is reasonable.

> The cost for Debian and it's mirror network is very high.

I'm curious how high it actually is. They say it's high, but that could well just be hand-waving. Sure, prior to things like LetsEncrypt those SSL certs would have been a notable financial burden. There's also some extra cost on infrastructure covering the cryptographic workload, but increasingly the processors in servers are capable of handling that without any notable effort.

Certificate cost is trivial. Let's encrypt makes it free, but with a small change to the host names (country code.ftp.debian.org instead of ftp.countrycode.debian.org), all mirrors could have been covered with a single certificate. Some CAs will let you buy one wildcard cert and issue unlimited duplicate certificates with the same name. So, that would cost some money, but probably not too much.

The real costs are organizational and technical.

Organizing all the different volunteers who are running the mirrors to get certificates installed and updated and configured properly is work. Maybe let's encrypt automation helps here.

From a technical perspective, assuming mirrors get any appreciable traffic, adding https adds significantly to the CPU required to provide service. TLS handshaking is pretty expensive, and it adds to the cost of bulk transfer as well.

I get the feeling that alot of the volunteer mirrors are running on oldish hardware that happens to have a big enough disk and a nice 10G ethernet. I've run a bulk http download service that enabled https, and after that our dual xeon 2690 (v1) systems ran out of CPU instead of out of bandwidth. CPUs newer than 2012 (Sandy Bridge) do better with TLS tasks, but mirrors might not be running a dual CPU system either.

Old hardware will eventually die and needs replacing. I run infrastructure for a CDN setup, and we actually _reeuced_ the CPU overhead with TLS 1.3 + HTTP/2.

When someone says the cost is high, most people jump to monetary issues. The cost is in the time and effort required to make the changes, and to have those changes synchronised across every single APT mirror.

None of your business.

But! As mentioned above, outside entities being able to monitor exactly which versions of which packages are being installed to which hosts is a significant security risk.

This sort of comments aren't helpful. Switching to HTTPS will require tremendous amount of work from volunteers. You need to convince me that (1) your usecase exists (2) your usecase can be remedied with HTTPS.

Nothing requires privacy. However, the HTTPS movement is about making privacy the default, not the exception.

The HTTPS movement is about ensuring that only Google gets your data. It's not about privacy.

I don't quite follow. Please explain why Google gets the data.

Presumably Google Analytics which can embed on HTTPS pages.

Which is already blocked by 40% of the users via ad-blockers.

Edit: some report even over 70% that block GA.

Using an adblocker is not part of HTTPS.

If I was a particular government, I could block apt requests for the tor package e.g.

We deploy our software packages to our own infrastructure and clients using a private APT repository and basic HTTP auth. Obviously we're running it with apt-transport-https installed for making the latter not completely insecure.

I see no reason to do that for signed packages from the main repositories, however.

For a theoretical example, hiding the fact that I have installed encryption or steganography from my government.

Genuinely: it's 2019: shouldn't privacy be the default?

yes but privacy is a whole another thing that is maybe not worth it. with mirrors and so on getting https to properly work is not trivial. sure it would be nice.

HTTPS is really quite trivial, especially with the advent of letsencrypt. This is especially true for simple package protocols like APT, where a repository is simply a dumb HTTP server coupled with a bunch of shell scripts that update the content.

Assuming that we consider SSH-ing into a server a negligible effort, then adding HTTPS to a APT repository or mirror is also a negligible effort.

As for whether privacy is worth it: Absolutely, especially in this day and age. There is very rarely a cost too high when it comes to privacy, and in this instance, it comes for free.

The problem is, HTTPS is not designed for privacy in any meaningful term.

1) TLS session negotiation leaks all sorts of useful data about both systems, not to mention TCP and IP stack on which it sits. This data is grabbed in 5 minutes with an existing firewall filter. Combined with IP, it shows the exact machine and web browser (incl. Apt version) downloading the file in many cases.

2) It does nothing to prevent time, host and transfer size fingerprinting.

3) Let's Encrypt helps with deployment but you get rotating automated server certificates. It is reasonably easy to obtain a fake Let's Encrypt certificate so without pinning it is worthless for authentication, pinning a rotating certificate is hard too.

Debian does not have resources to handle impostor mirrors.

it's not trivial if we are talking about Linux boxes serving as servers let's encrypt has a good chance to not work out of the box, and especially with older boxes. and then there i are other things like needing a http server for obtaining the cert rotating it, distributing it. and you loose the ability to use a proxy, and so on. with https you are still not protected with them knowi g where you get only what you did there.

it would be great to have the ability to have https but for APT in its current form and for what it is used the cost benefit for adding https is not that compelling to me.

Why would it? As the site lays out, HTTPS is strictly inferior to the scheme used by APT.

You can not get more security by adding a less secure mechanism to a better one. It's not additive.

Of course you'd get more security.

Analogy: It's like your hanging on a rope and also add a safety net. If the rope breaks, you only fall on the safety net instead of the ground.

All software has bugs, so if you add two buggy security solutions, you might only be able to exploit a bug in one, but the other still gives you the safety.

That's not always true though, and it can be really hard to tell if a particular combination is going to make things better or worse. e.g. the "Breach" HTTPS+Compression vulnerability.

To continue your analogy: The rope gets tangled in the safety net, forcing you to jump or proceed up with a loose rope, because you can no longer move the rope..

This is not the physical world.

Digital security strength is measured on orders of magnitude, and two mechanisms providing security with very different orders of magnitude do not add in any practical sense.

If you've seen today's bug yesterday, or carefully looked at the previous CVEs, you would have seen that https would have significantly reduced the probability of exploitability.

> All software has bugs, so if you add two buggy security solutions, you might only be able to exploit a bug in one, but the other still gives you the safety.

I am unconvinced about the last part. More commonly an exploit in either will cause security to fail, so adding more steps just adds more attack surface and leads to less security.

It depends on the threat model, here TM1 and TM2:

TM1: attacker does not posses zerodays to installed software

TM2: attacker possesses speccific (perhaps OS, perhaps library, perhaps userland) zerodays, usage of which (including unsuccesful attempts) should be minimized to avoid detection

in TM1: it's ok to use HTTP in the clear, as long as signatures are verified

in TM2: everything should be fetched over encrypted HTTPS, since HTTP would leak information about available attack surface

EDIT: not only would this increase security by not revealing what a user installs (perhaps download some noise as well such that it becomes harder to detect what a user is installing?), it could also improve security by turning the APT servers into honeypots, so that monitoring these can reveal zerodays...

TM3: attacker has a 0-day against a complex https server and they replace packages on some mirrors.

TM4: attacker can impersonate a server using Lets Encrypt certificate and bypass their automated verification, creating a fake mirror or a bunch. (HTTP has same vector.) They can also make DNS fail or reroute.

TM5: attacker has a 0-day against the more complex https client (e.g. curl).

TM6: Attacker fingerprints network connections to given servers by size and os or os + tls fingerprinting data

TM3 and TM5 are specific subcases for TM2, and turn the APT server / client into a honeypot

TM6: we should have an overlay onion router, and I agree that the current complexity is worrying, I'd love to see a minimalist version of TOR (with minimal I don't necessarily mean the code size should be small, but minimal assumptions, and that the safety of the system can be verified from the assumptions)

TM4: I don't understand, Lets Encrypt does not calculate private keys for public keys...

> if APT was designed today I'm sure it would use HTTPS.

Or it would use the much more modern and more secure Noise, like I think QUIC will end up using through nQUIC:

https://dl.acm.org/citation.cfm?id=3284854

https://noiseprotocol.org

Yeah, no.

nQUIC is maybe an interesting approach for some specialist applications but more likely a dead end.

The idea you can replace QUIC with nQUIC is like when Coiners used to show up telling us we're going to be using Bitcoin to buy a morning newspaper. Remember newspapers?

nQUIC doesn't have a way for Bob to prove to Alice that he's Bob beyond "Fortunately Alice already knew that" which is the assumption in that ACM paper. So that's a non-starter for the web.

nQUIC also doesn't have a 0-RTT mode. Noise proponents can say "That's a good thing, 0-RTT is a terrible idea". Maybe so, but you don't have one and TLS does. If society decides it hates 0-RTT modes because they're a terrible idea, we just don't use the TLS 0-RTT mode and nothing is lost. But if as seems far more likely we end up liking how fast it is, Noise can't match that. Doesn't want to.

Noise is a very applicable framework for some problems, and I can see why you might think APT fits but it doesn't.

> nQUIC is maybe an interesting approach for some specialist applications but more likely a dead end.

Adding on to this, nQUIC (and Noise specifically) is significantly better for use-cases where CAs and traditional PKI don't make sense, e.g., p2p, VPN, TOR, IPFS, etc...

I agree that APT is not one of these cases. Currently APT has a root trust set that is disjoint from the OS's root CA set, but they could easily do HTTPS and just explicitly change the root CA set for those connections.

EDIT: from the nQUIC paper:

> In particular nQUIC is not intended for the traditional Web setting where interoperability and cryptographic agility is essential.

On another note, I think it would be helpful to expand some points for other readers:

> nQUIC also doesn't have a 0-RTT mode. Noise proponents can say "That's a good thing, 0-RTT is a terrible idea".

0-RTT is dangerous because of replay attacks. It pushes low-level implementation details up the stack and requires users to be aware of and actively avoid sending non-idempotent messages in the first packet.

> Maybe so, but you don't have one and TLS does. If society decides it hates 0-RTT modes because they're a terrible idea, we just don't use the TLS 0-RTT mode and nothing is lost.

One major point of using Noise protocol is to _simplify_ the encryption and auth layers, remove everything that's not absolutely necessary, and make it hard to fuck up in general. Things like ciphersuite negotiation, x509 certificate parsing and validation, and cryptographic agility have been the source of many many security critical bugs.

From an auditability perspective, Noise wins easily. You can write a compliant Noise implementation in <10k loc, vs. OpenSSL ~400k loc.

> But if as seems far more likely we end up liking how fast it is, Noise can't match that. Doesn't want to.

HTTP is insecure, but faster than HTTPS. Most sites now use HTTPS regardless. 0-RTT is insecure and while it might be OK for browsing HN, removing 0-RTT makes it much harder to fuck up.

>I can see that they don't want to make demands on a service they get for free, when HTTPS isn't even necessary for the use case

you could imagine a situation where https would be optional for APT mirrors. Then the package manager would have a config flag to use any mirror or only https-enabled mirrors (probably enabled by default). This would allow to use https without creating any demands to organizations that host those mirrors - if they can they would enable it, but it would not be required. The https-enabled hosts could also provide plain http for backwards compatibility.

The argument isn’t correct, what does a user do when the download is damaged by an injection? A re-download results in exactly the same tampered with file.