News for hackers. Stuff that matters.
Re:Why does APT not use HTTPS? by hannob on Monday January 21, 2019 @11:04AM (#18961209)
I guess I can copy over a comment I made when this previously made rounds:

I have a few problems with this. The short summary of these claims is “APT checks signatures, therefore downloads for APT don’t need to be HTTPS”.

The whole argument relies on the idea that APT is the only client that will ever download content from these hosts. This is however not true. Packages can be manually downloaded from packages.debian.org and they reference the same insecure mirrors. At the very least Debian should make sure that there are a few HTTPS mirrors that they use for the direct download links.

Furthermore Debian also provides ISO downloads over the same HTTP mirrors, which are also not automatically checked. While they can theoretically be checked with PGP signatures it is wishful thinking to assume everyone will do that.

Finally the chapter about CAs and TLS is - sorry - baseless fearmongering. Yeah, there are problems with CAs, but deducing from that that “HTTPS provides little-to-no protection against a targeted attack on your distribution’s mirror network” is, to put it mildly, nonsense. Compromising a CA is not trivial and due to CT it’s almost certain that such an attempt will be uncovered later. The CA ecosystem has improved a lot in recent years, please update your views accordingly.

Re:Why does APT not use HTTPS? by omarforgotpwd on Monday January 21, 2019 @03:58PM (#18963555)
The other big problem is that people can see what you're downloading. Might not be a big deal but consider:

1. You're in China and you download some VPN software over APT. A seemingly innocuous call to package server is now a clear violation of Chinese law.

2. Even in the US, can leak all kinds of information about your work habits, what you're working on, etc.

3. If it's running on a server, it could leak what vulnerable software you have installed or what versions of various packages you're running to make exploiting known vulnerabilities easier

Re:Why does APT not use HTTPS? by jlarocco on Monday January 21, 2019 @05:32PM (#18964262)
Even with HTTPS, it's easy enough to figure out which package is being downloaded based on the download size, as explained in TFA.

Every time this comes up, it's always the same handful of incorrect arguments made in favor of HTTPS.

The cargo-cult mentality that HTTPS == security really does more damage than it does good.

Re:Why does APT not use HTTPS? by zaptheimpaler on Monday January 21, 2019 @06:04PM (#18964454)
I mean deducing the package from download size is way harder than just seeing the name in the open.. security is rarely perfect and more like an arms race, making things more difficult is a big deal. This kind of "you can hack Y too" argument doesn't make any sense if its way harder to hack Y than X.
Re:Why does APT not use HTTPS? by jlarocco on Monday January 21, 2019 @06:23PM (#18964592)
I would agree with you IF it were actually "way harder", but realistically it's no more difficult than checking the file name or the md5 checksum.
Re:Why does APT not use HTTPS? by gtsteve on Tuesday January 22, 2019 @07:44AM (#18967844)
What if keepalive is used and multiple packages are downloaded? Surely that is at least plausible deniability.
Re:Why does APT not use HTTPS? by wallunit on Monday January 21, 2019 @06:34PM (#18964659)
Is that even true? In practice rather than downloading a single package you'd download/update a bunch of packages over the same connection, and an attacker would only see the accumulated size, right?
Re:Why does APT not use HTTPS? by jlarocco on Monday January 21, 2019 @06:51PM (#18964765)
You can see when you run "apt-get install ..." or "apt-get upgrade" that it opens multiple connections to download packages...

And the Debian contributor who wrote TFA says it's possible, and I'm sure he knows a lot more about it than I do.

Re:Why does APT not use HTTPS? by pvorb on Monday January 21, 2019 @06:50PM (#18964752)
I'm not sure how APT handles connections, but with a typical browser connections will be reused if requests are made shortly after another.

That doesn't mean it's impossible to determine what packages you downloaded. But it will be more effort to do so.

Re:Why does APT not use HTTPS? by dcbadacd on Monday January 21, 2019 @08:45PM (#18965311)
Less with HTTP2/TLSv1.3, endpoints are also hard to detect with ESNI.
Re:Why does APT not use HTTPS? by AstralStorm on Tuesday January 22, 2019 @02:59AM (#18966741)
No they aren't. HTTPS fingerprinting is easy. It's been done by lcamtuf years ago and it's available as a layer 7 filter in Linux... TLS adds more information because it prevents proxies and has specific server implementations.
Re:Why does APT not use HTTPS? by loeg on Monday January 21, 2019 @06:09PM (#18964498)
This is addressed in the "privacy" section. Basically, your premise is wrong. TLS does not provide additionally privacy for this usecase. The short summary is: the size of transmitted data makes inferring what you've downloaded from a public file mirror trivial for a passive observer, even with TLS.
Re:Why does APT not use HTTPS? by hopler on Monday January 21, 2019 @07:46PM (#18965025)
Perhaps apt should pad its downloads
Re:Why does APT not use HTTPS? by JdeBP on Tuesday January 22, 2019 @07:27AM (#18967771)
This has already been discussed in this very discussion.

* https://news.ycombinator.com/item?id=18959470

Re:Why does APT not use HTTPS? by loeg on Monday January 21, 2019 @07:57PM (#18965085)
Padding doesn't solve the privacy problem without unacceptable tradeoffs (i.e., pad all packages to the same size).
Re:Why does APT not use HTTPS? by sattoshi on Monday January 21, 2019 @08:52PM (#18965361)
Pad all packages to the nearest in a list of sizes?
Re:Why does APT not use HTTPS? by loeg on Monday January 21, 2019 @09:56PM (#18965653)
How many bits of privacy are you willing to give up here? Debian only has about 48,000 packages. That's almost 16 bits, total, with perfect privacy — all packages enlarged to the same size.

You can select a list of sizes trading off collisions (2 packages with same size => 1 bit of privacy; 4 => 2 bits, etc). But the most you ever get is (nearly) 16. The amount of padding you need to even get 2 bits of privacy (giving up almost 14) on the long tail of large packages is going to be "a lot" and it grows as you want more bits.

Re:Why does APT not use HTTPS? by shasheene on Tuesday January 22, 2019 @04:54AM (#18967202)
Some further pros and cons of padding is discussed elsewhere in the thread.

Providing privacy for the packages that, including dependencies, are less than 100MB in size is something that's probably worth doing. The cost of padding an apt-get process to the nearest say, 100MB, is not necessarily infeasible as far as bandwidth goes.

Instead of padding individual files, how about a means to arbitrarily download some number of bytes from an infinite stream? That would appear to be sufficient to prevent file size analysis (but probably not timing attacks).

Exposing something like /dev/random via a symbolic link and allowing the apt-get client to close the stream after the total transfer reaches 100MB would appear to make it harder to infer packages based on the transferred bytes, without being very difficult to roll out.

Re:Why does APT not use HTTPS? by dcbadacd on Monday January 21, 2019 @08:58PM (#18965398)
It's a big step from "just `cat` the traffic to know" to "start comparing file-sizes and hope no sizes match" in terms of privacy.
Re:Why does APT not use HTTPS? by JdeBP on Tuesday January 22, 2019 @07:27AM (#18967770)
Re:Why does APT not use HTTPS? by loeg on Monday January 21, 2019 @09:58PM (#18965661)
I'd say you're flat out wrong — there is no big step. We'll have to agree to disagree, I guess.
Re:Why does APT not use HTTPS? by omarforgotpwd on Monday January 21, 2019 @05:10PM (#18964102)
Oh, another thing I just thought of: It could leak what vulnerable software you have installed / what version you're running to make exploits easier.
Re:Why does APT not use HTTPS? by NicoJuicy on Monday January 21, 2019 @01:22PM (#18962259)
Here is why Videolan (VLC) doesn't do it: https://www.beauzee.fr/2017/07/04/videolan-and-https/

There were some topics about it yesterday.

Eg. https://news.ycombinator.com/item?id=18948195

Some arguments of the blog post are also valid here

Re:Why does APT not use HTTPS? by hacknat on Monday January 21, 2019 @11:19AM (#18961332)
Also, their claim that HTTPS doesn’t hide the hosts that you are visiting is about to not be true. Encrypted SNI is now part of the TLS 1.3 RFC, so HTTPS will actually hide one’s internet browsing habits quite well. The only holes in privacy left on the web’s stack are in DNS.
Re:Why does APT not use HTTPS? by toast0 on Monday January 21, 2019 @11:39AM (#18961470)
Can you point out where encrypted SNI is in the RFC? I've read the RFC, and I don't recall it being in there. I do see that there is an extension published, which I haven't reviewed in depth.

From a breif review, I see two potential issues:

a) the encrypted sni record contains a digest of the public key structure. This digest is transmitted in the clear (as it must be at this phase of the protocol), so a determined attacker could create a database of values for the top N mirror sites.

b) in order to be useful, the private key for the public keys would need to be shared across all servers supporting that hostname. That's not a big deal for a normal deployment, but it's not great for a volunteer mirrors system -- lots of diverse organizations own and operate the individual mirrors and we need to count on all of those to keep it secure. Also, it adds an extra layer of key management, which is an organizational and operational burden.

Re:Why does APT not use HTTPS? by tialaramex on Monday January 21, 2019 @02:37PM (#18962920)
Yeah, your parent is wrong about it being in the RFC. ESNI is something that they decided wasn't possible and ruled out of scope for TLS 1.3 but then somebody had a brainwave and Rescorla plus some people at Cloudflare wrote IDs and did live fire testing. The drafts are maybe at the "this is the rough shape of a thing" stage, more than ambitions but not a basis on which to announce specific plans.

It's also pointless without DPRIVE. If people can see all your DNS lookups they can guess exactly what you're up to. That's why that Firefox build did both eSNI and DoH

Re:Why does APT not use HTTPS? by topranks on Tuesday January 22, 2019 @08:23PM (#18974799)
Yeah it’s not part of the TLS 1.3 spec (RFC8446).

It’s being added, but early stages. Internet draft is here:

https://www.ietf.org/id/draft-ietf-tls-esni-02.txt

Re:Why does APT not use HTTPS? by ec109685 on Monday January 21, 2019 @11:33AM (#18961430)
For sni to be effective, you need to be hitting hosts that serve more than one domain. Otherwise, you can just reverse map the ip.
Re:Why does APT not use HTTPS? by y4mi on Monday January 21, 2019 @11:44AM (#18961508)
Most of my severs don't report the actual service dns back on a reverse lookup. It's generally nodeXX.some.fqdn or clusterXX or lbXX
Re:Why does APT not use HTTPS? by shittyadmin on Monday January 21, 2019 @12:27PM (#18961854)
Doesn't really matter either way - there's a bunch of crawlers and scanners out there such that you can pretty much Google any IP and find a list of sites that are hosted on it.
Re:Why does APT not use HTTPS? by dcbadacd on Monday January 21, 2019 @08:46PM (#18965317)
Fun fact, most of those break if you just close the connection when the client doesn't support SNI.
Re:Why does APT not use HTTPS? by shittyadmin on Tuesday January 22, 2019 @12:21AM (#18966161)
Not quote the ones I was referring to - many services just look at DNS and get the A record for every domain then offer reverse lookup - complete lists of domains are purchasable for all major TLDs. The only defense to this would be to host your content on a subdomain.

DNSlytics, DomainTools, W3Advisor and others offer this.

Re:Why does APT not use HTTPS? by Ayesh on Tuesday January 22, 2019 @12:42AM (#18966233)
It's also easy to just scan a whole range (say, top 1m Alexa domains) and log the IP. You can scan 1m sites on a cheap $5 VPS nowadays.
Re:Why does APT not use HTTPS? by jedberg on Monday January 21, 2019 @12:21PM (#18961806)
Yes but if I want to specifically look for traffic to the Debian mirrors, I can use DNS to build a list of the IPs and then see if you're connecting to one of them.
Re:Why does APT not use HTTPS? by pbhjpbhj on Monday January 21, 2019 @01:27PM (#18962301)
Most services report what they are, and even the server often, when you connect. If you connect to an IP and it's serving a website then I don't know why you'd care that reverse-lookup isn't configured correctly, you're not hiding anything?
Re:Why does APT not use HTTPS? by admax88q on Monday January 21, 2019 @01:43PM (#18962432)
Its feasible to build a reverse lookup table of all registered domain names.
Re:Why does APT not use HTTPS? by zamadatix on Monday January 21, 2019 @11:26AM (#18961389)
I bet DoHTTPS/DoTLS has more penetration than ESNI. Neither are widely implemented though, which is the important part.
Re:Why does APT not use HTTPS? by da_chicken on Monday January 21, 2019 @01:19PM (#18962243)
Yeah, but not being implemented now doesn't mean that they never will be. Migrating/implementing now will mean that the mirrors will automatically support it once the features are in place and the clients and servers can agree on the improved privacy feature sets.
Re:Why does APT not use HTTPS? by progval on Monday January 21, 2019 @11:37AM (#18961458)
By host, they probably mean the IP address.
Re:Why does APT not use HTTPS? by jdamato on Monday January 21, 2019 @12:39PM (#18961949)
Thanks for commenting this! I've seen this website before and it's really unfortunate how much attention it gets.

APT's use of plain text HTTP (even with GPG) is vulnerable to several attacks outlined in this paper: https://isis.poly.edu/~jcappos/papers/cappos_mirror_ccs_08.p....

Yes, this paper is old, but APT is still vulnerable to most of these attacks. I would advise anyone wanting to use APT to do so only with TLS.

Re:Why does APT not use HTTPS? by loeg on Monday January 21, 2019 @06:17PM (#18964553)
The criticisms in that paper either do not apply to Apt as described in TFA or amount to DoS attacks. HTTPS does not and can not solve DoS.
Re:Why does APT not use HTTPS? by Spooky23 on Monday January 21, 2019 @04:46PM (#18963904)
For a mirror based system like apt it is incredibly trivial. The integrity of a package depends on dozens of organizations and their practices.

When I was a 19 year old idiot, I was responsible for a mirror server. As a bad actor, I could easily get access to a valid organizational cert.

Re:Why does APT not use HTTPS? by sanxiyn on Monday January 21, 2019 @05:25PM (#18964214)
> When I was a 19 year old idiot, I was responsible for a mirror server.

Me too! It was even ftp.kr.debian.org! It still is!

Seriously, people, who do you think has root of official Debian mirror servers hosted by universities? University students. Who are 19 years old. This is literally true.

Re:Why does APT not use HTTPS? by Ayesh on Tuesday January 22, 2019 @12:48AM (#18966257)
In my country, the ccTLD registry is run by a university. While the professors have done an excellent job, the NIC itself was hacked a few times back, there is no admin UI (call a 19 year old kid and set your nameservers with NATA phonetics), and they still have some non functional root nameservers.
Re:Why does APT not use HTTPS? by simias on Monday January 21, 2019 @12:45PM (#18961989)
>Packages can be manually downloaded from packages.debian.org and they reference the same insecure mirrors.

That's a reasonable complaint. I think it would make sense for the individual packages to be signed as well (and checked during install). This way you'd get a warning if you install an untrusted package regardless of the source. I'm not sure why it doesn't work that way.

>Furthermore Debian also provides ISO downloads over the same HTTP mirrors, which are also not automatically checked. While they can theoretically be checked with PGP signatures it is wishful thinking to assume everyone will do that.

I do do that. If you care about such an attack vector why wouldn't you? And if you don't, why should Debian care for you? There are plenty of mirrors for Debian installers, I often get the mine from bittorent, trusting a PGP sig makes much more sense than relying on HTTPS for that IMO.

>Finally the chapter about CAs and TLS is - sorry - baseless fearmongering.

I don't think it's baseless (we have a long list of shady CAs and I'm sure may government agencies can easily generate forged certificates) but it's rather off-topic. Their main argument is that the trust model of HTTPs doesn't make sense for APT, if that's true then whether or not HTTPS is potentially hackable is irrelevant.

Re:Why does APT not use HTTPS? by driverdan on Monday January 21, 2019 @01:22PM (#18962267)
> If you care about such an attack vector why wouldn't you? And if you don't, why should Debian care for you?

Security should be as automatic as possible. It should be assumed that any step that requires manual intervention will be skipped by most people.

Re:Why does APT not use HTTPS? by simias on Monday January 21, 2019 @01:47PM (#18962480)
But the point the page makes is that HTTPS wouldn't be good enough anyway. As such it's not a replacement for checking the PGP signature. I think it's consistent.

If HTTPS could be used to replace PGP signature checks then I'd agree with you but it's not the case. So I go back to my initial point, if you worry about your image being tampered with HTTPS is not enough. If you don't care then you don't care either way.

In a way not using HTTP is kind of an implicit disclaimer on Debian's part. "Don't trust what you get from this website". If they feel like they can't guarantee the security of whatever server is hosting the CD images adding HTTPS might actually be a bad thing because people who might otherwise have checked the signature may think "well, it's over HTTPS, it's good enough".

Re:Why does APT not use HTTPS? by antris on Monday January 21, 2019 @01:32PM (#18962344)
>It should be assumed that any step that requires manual intervention will be skipped by most people.

Indeed.

1. If you don't care about security, it still doesn't hurt to have HTTPS. Think of it as "extra" that you get for free.

2. If you care about security, you might still don't have the know-how to make sure everything is secure and don't have time to get into it as you're trying to get things done.

3. Even if you care about security AND have the know-how, you might still forget. Nobody's perfect. So it's good that the HTTPS is there.

Re:Why does APT not use HTTPS? by wolco on Monday January 21, 2019 @03:43PM (#18963444)
Nothing is for free, https has additional costs over http. In many cases it makes sense to pay those costs but let's not forget about them.
Re:Why does APT not use HTTPS? by bigiain on Monday January 21, 2019 @06:54PM (#18964780)
To be fair to the apt developers/maintainers - the security _is_ automatic when using their tool to talk to their repos.

It's not their responsibility to automate security for people using their repos via different tools.

If the solution was just "install certbot on the server and use a free https cert" then perhaps you could make an argument saying maybe they should just do it. But when the problem space includes aggressively using a global (largely volunteer) mirror network and supporting local caching proxies, I can completely understand why they'd say "Nope. Not our problem, not our responsibility to provide a solution. We've got other more productive ways to spend our and our mirror volunteers time and effort".

Re:Why does APT not use HTTPS? by Conan_Kudo on Monday January 21, 2019 @06:43PM (#18964714)
> I think it would make sense for the individual packages to be signed as well (and checked during install). This way you'd get a warning if you install an untrusted package regardless of the source. I'm not sure why it doesn't work that way.

Actually, debs are not signed in Debian or Ubuntu. The accepted practice in Debian is that only repository metadata is signed.

The argument is that the design of Debian packages (as in, the package format) makes it difficult to reproducibly validate a deb, add a signature, strip the signature, and validate it again.

Personally, I'm not sure I buy it, as we don't have problems signing both RPMs and RPM repository metadata. Technically, yes, the RPM file format is structured differently to make this easier ('rpm' is a type of 'cpio' archive), but Debian packages are fundamentally 'ar' archives with tarballs inside, and those aren't hard to do similar things. For reproducible builds, Koji (Fedora's build system) and OBS (Open Build Service, openSUSE's build system) are able to strip and add signatures in a binary-predictable way for RPMs.

Fedora goes the extra step of shipping checksums via metalink for the metadata files before they are fetched to ensure they weren't tampered before processing. But even with all that, RPMs _are_ signed so that they can be independently verified with purely the usage of 'rpm(8)'.

Re:Why does APT not use HTTPS? by oefrha on Monday January 21, 2019 @01:17PM (#18962224)
> And if you don't, why should Debian care for you?

Those who don’t go out of their way to defend themselves don’t deserve security — seriously, that’s your attitude? Then you don’t deserve to be in any position to make security decisions for other people.

Re:Why does APT not use HTTPS? by 0xdeadbeefbabe on Monday January 21, 2019 @05:58PM (#18964424)
Security isn't on or off. If you think it is then "you don't deserve [...]" It seems like reasonable trade off.
Re:Why does APT not use HTTPS? by emilfihlman on Monday January 21, 2019 @12:59PM (#18962098)
>Compromising a CA is not trivial and due to CT it’s almost certain that such an attempt will be uncovered later. The CA ecosystem has improved a lot in recent years, please update your views accordingly.

This is simply not true. Governments can simply compel your CA to do as they want. Not to mention that "uncovered later" is pretty damn worthless.

That said, I do agree that there should be HTTPS mirrors.

Re:Why does APT not use HTTPS? by electrum on Tuesday January 22, 2019 @12:53AM (#18966279)
> Not to mention that "uncovered later" is pretty damn worthless.

It is not worthless as a deterrent to the CA. Proof of a fraudulently issued certificate are grounds to permanently distrust the CA. So, yes, they can do it, but hopefully only once.

Re:Why does APT not use HTTPS? by pbhjpbhj on Monday January 21, 2019 @01:30PM (#18962331)
Can't Debian devs just hookup their own server as a CA? No need to trust anyone outside the Debian universe.

In fact, is the FOSS community running it's own trust network still, it used to be a thing at LUGs.

Re:Why does APT not use HTTPS? by notyourday on Monday January 21, 2019 @01:22PM (#18962264)
The justifications for why APT does not use HTTPS ( by default, it is possible to add https transport ) are just mind blowing. It is however not at all surprising considering how broken Debians secure package distribution methodology is -- I'm saying it as someone who had to implement workarounds for it for a company that was willing to spend significant amount of resources on making it work.

Here's are some low level gems:

1. I have installed package X. I want to validate that the files that are listed in a manifest for package X have not changed on a host.

APT answer: Handwave! This is not a valid question. If you are asking this question you already lost.

2. I want to have more than one version of a package in a distribution.

APT answer: Handwave! You don't need it. You can just have multiple distributions! It is because of how we sign things - we sign collections!

3. I want to have a complicated policy where some packages are signed, some are not signed, and some are signed with specific keys.

APT answer: Handwave! You should have all or nothing policy! Nothing policy, actually, because we mostly just sign collections, rather than the individual packages

https://wiki.debian.org/SecureApt

Re:Why does APT not use HTTPS? by geezerjay on Monday January 21, 2019 @03:14PM (#18963233)
> 2. I want to have more than one version of a package in a distribution.

> APT answer: Handwave! You don't need it. You can just have multiple distributions! It is because of how we sign things - we sign collections!

This is not true at all. If you need to distribute multiple versions of the same package then all you need to do is provide multiple versions of the same package. Just get your act straight, learn how to package software, create your packages so that you can deploy them simultaneously without breaking downstream software, and you're set.

Re:Why does APT not use HTTPS? by drdaeman on Monday January 21, 2019 @06:18PM (#18964557)
AFAIK, dpkg doesn't allow to have multiple versions of the same package installed at the same time.

This is not apt's limitation, though.

Re:Why does APT not use HTTPS? by geezerjay on Tuesday January 22, 2019 @09:49AM (#18968587)
> AFAIK, dpkg doesn't allow to have multiple versions of the same package installed at the same time.

That's true, but multiple versions of the same package is not the same as multiple versions of the same software. Debian and Ubuntu have access to multiple minor version releases of GCC and they can all coexist side by side.

Re:Why does APT not use HTTPS? by UncleEntity on Monday January 21, 2019 @07:31PM (#18964960)
Too lazy to get up from my comfy easy chair to boot my debian running NAS but 99.9% sure there's more than one version of python on that thing.
Re:Why does APT not use HTTPS? by notyourday on Monday January 21, 2019 @08:27PM (#18965222)
No, those are <somepackage>-<someversion> where <somepackage> is different.
Re:Why does APT not use HTTPS? by Conan_Kudo on Monday January 21, 2019 @08:57PM (#18965389)
Correct. Multi-version packages are not supported by dpkg. They are supported by rpm in limited circumstances (no file conflicts allowed!). Fedora and openSUSE kernel packages work this way, as an example.

The way Debian works around this is by doing "<name>-<version>" as the package name. This is a valid approach, though it makes package name discovery a bit more difficult at times...

As for multiple versions of packages in the repo, "createrepo"/"createrepo_c" (for RPM repositories) does not care.

And for Debian, "dpkg-scanpackages" can be made to not care too, using the "--multiversion" switch: https://www.mankier.com/1/dpkg-scanpackages#--multiversion

This is somewhat at your peril, as I've observed APT getting confused when it parses metadata from repositories produced by dpkg-scanpackages that allows multiple versions in the repository.

However, reprepro does not support this at all, so most deployments with semi-large Debian repositories will not have this option available to them anyway.

Re:Why does APT not use HTTPS? by geezerjay on Tuesday January 22, 2019 @09:40AM (#18968506)
> No, those are <somepackage>-<someversion> where <somepackage> is different.

It seems there is a significant semantics gap between how deb packages work and what's supposed to be a package version.

In deb packages, package versions are lexicograhically ordered descriptions of a version ID that is used to guide autoupgrades.

If a packager wishes that multiple minor version releases should be present in a system then he should build his packages to reflect that, which is exactly how Python packages, and specially some libraries, do. For example, python packages are independent at the major version level but GCC packages are independent at the minor version level.

Re:Why does APT not use HTTPS? by notyourday on Tuesday January 22, 2019 @12:32PM (#18970197)
> If a packager wishes that multiple minor version releases should be present in a system then he should build his packages to reflect that, which is exactly how Python packages, and specially some libraries, do. For example, python packages are independent at the major version level but GCC packages are independent at the minor version level.

Not in a system. In a repo. Standard debian tools ( not hacked, not outside the tree, not outside the debian main tree ) do not support this.

If you have a repo with <packagename>-<packageversion> and add a package <packagename>-<packageversion1> where packageversion1 is higher than the package version, the previous version gets deleted from the repo.

Can it be worked around? Sure, you can:

a) have multiple repos. If you want to keep up to a hundred versions, you can just create a hundred repos.

b) you can redefine the meaning of a package name and incorporate the version into the name of the package.

(b) Sounds like a good solution. Except if the org decided to use something like .deb for distribution of artifacts it is probably the org that uses other software. Let's say it uses puppet, which supports Debian package management out of the box, except that now you need to change how puppet uses version numbers because as we started embedding the version name into the package name "nginx-1.99.22" and "nginx-1.99.23" became two different packages not two different versions of the same package.

This goes on and on.

Re:Why does APT not use HTTPS? by geezerjay on Tuesday January 22, 2019 @01:44PM (#18971010)
> Not in a system. In a repo. Standard debian tools ( not hacked, not outside the tree, not outside the debian main tree ) do not support this.

That's the semantic gap I've mentioned.

Your statement is patently wrong, as deb packages do enable distributions such as Debian and other Debian-based distros such as Ubuntu to provide multiple major, minor, and even point releases through their official repos to be installed and run side-by-side .

Take, for example, GCC. Debian provides official deb packages for multiple major and minor release versions of GCC, and they are quite able to coexist in the very same system.

All it takes for someone to build deb packages for multiple versions of a software package is to get to know deb, build their packages so that they can coexist, and set their package and version names accordingly.

Re:Why does APT not use HTTPS? by notyourday on Tuesday January 22, 2019 @04:32PM (#18972738)
> Your statement is patently wrong, as deb packages do enable distributions such as Debian and other Debian-based distros such as Ubuntu to provide multiple major, minor, and even point releases through their official repos to be installed and run side-by-side .

Through multiple repos. Not through one repo. That's why testing packages live in a separate repo. That's why security packages live in a separate repo. That's why updates live in a separate repo.

If you are using careful versioning on packaging avoiding the <packagename>-<version> as the convention you are breaking other tools, including the tools that are distributed with Debian. Puppets' package { "nginx": ensure => installed } will not work if you decided to allow nginx to have multiple versions using "nginx-1.99" as a special package name.

Re:Why does APT not use HTTPS? by geezerjay on Wednesday January 23, 2019 @05:54AM (#18977234)
> Through multiple repos.

Wrong. The official debian package repository hosts projects which provide multiple major and even minor versions of the same software package to be installed independently and to coexist side-by-side.

Seriously, you should get to know debian and its packaging system before making any assertion about them. You can simply browse debian's package list and search for software packages such as GCC to quickly acknowledge that your assertion is simply wrong.

Re:Why does APT not use HTTPS? by notyourday on Wednesday January 23, 2019 @07:59AM (#18977749)
It is not. You are talking about a hack/workaround/different methodology. That hack cannot be natively integrated with the other software that uses a regular idea of what is a package name and what is a package version (for example, puppet, which ships with Debian). It is a garbage approach just like the APT approach of using a repo manifest for signatures is garbage, and just like approach of not storing crypto hashes of every file in a .deb inside the .deb is garbage.

I have provided the challenge in another reply:

I have in a repo:

package: nginx version: 1.99.77

I need to add to the repo:

package: nginx version: 1.99.78

Both of the versions must remain in the repo. Packages must be signed. Repo must be signed. The name of the package cannot change and neither can the version number. Both of the packages must be installable using a flag that specifies a version number passed to one of the standard Debian package install tools (the tool must be in "main" collection )

What is a tool that can be used that is listed https://wiki.debian.org/DebianRepository/Setup as existing in the "main" part of the Debian? For half a point, you can use any tool listed in the wiki.

Everything else is the hand waving.

Re:Why does APT not use HTTPS? by notyourday on Monday January 21, 2019 @08:27PM (#18965225)
repropro has that limitation.
Re:Why does APT not use HTTPS? by notyourday on Monday January 21, 2019 @08:26PM (#18965221)
Been there. Done that. It does not work.

I want to have

nginx-1.99.77 nginx-1.99.78 nginx-1.99.78ehjkhk-1

in the same repo. It is not possible.

Re:Why does APT not use HTTPS? by glennpratt on Tuesday January 22, 2019 @03:29AM (#18966851)
Unless I'm missing something, this is 100% possible. Now, if you want to carefully control the version constraints of these, welcome to pinning hell, but that's a different problem.
Re:Why does APT not use HTTPS? by notyourday on Tuesday January 22, 2019 @08:43AM (#18968157)
I'm always willing to see how to do it. The repo currently contains

<thispackage>-<this-version>-<this-patchlevel>.architecture

The package name is <thispackage>

I need to add to the repo <thispackage>-<this-version>-<that-patchlevel>.architecture

Packages are signed and repo will be signed. At the end I should be able to install the package using the select the specific version flag without adding another repo.

Re:Why does APT not use HTTPS? by glennpratt on Thursday January 24, 2019 @03:43PM (#18992727)
This is possible, if you are having issues, it is likely you apt repo tooling, not apt itself.

I use aptly and rely on this everyday.

Re:Why does APT not use HTTPS? by JdeBP on Tuesday January 22, 2019 @07:50AM (#18967865)
Rubbish. I have every version of the nosh toolset packages from version 1.22 to 1.39 side by side in a single Debian repository.
Re:Why does APT not use HTTPS? by notyourday on Tuesday January 22, 2019 @08:39AM (#18968127)
Re:Why does APT not use HTTPS? by JdeBP on Wednesday January 23, 2019 @09:25AM (#18978321)
A listing of all of these packages side by side, along with information on how it is built that includes the very scripts used to do so (including constructing that listing), is one of the value-added bonuses in the GOPHER version of the repository:

* gopher://jdebp.info:70/1/Repository/debian/dists/stable/

* https://news.ycombinator.com/item?id=14837740

Re:Why does APT not use HTTPS? by notyourday on Wednesday January 23, 2019 @10:12AM (#18978754)
Ok, so just to make it sure I understand:

1) It is not listed in the debian wiki

2) To access the "methods" i need to install a gopher client

That's pretty much a definition of the "handwave! Workaround!"

P.S. I have implemented a workaround. It works. It is just not a solution. I should not need to assign people to resign the packages with our own keys and create utilities that would duplicate Debian tools to provide a nominal access to regular required functionality by a large organization that has packages with hundreds of dependencies and could have 5-20 versions of the apps in different production/test/validation/qa environments. Joe Random Engineer expects that his knowledge of how apt-get works, how dpkg works and how puppet works to be portable.

Re:Why does APT not use HTTPS? by JdeBP on Wednesday January 23, 2019 @10:52AM (#18979229)
You clearly do not understand, especially given that gibberish about not being listed in some wiki, and utter irrelevancies about dpkg and puppet, which have no bearing upon the matter of publishing one Debian repository with multiple versions of packages at all.

I'm just not going to laboriously re-type it all into Hacker News when you can just go to the published repository and see it all explicitly laid out right there in the repository itself, with the exact scripts and commands that get run to produce the very repository that you are seeing -- quite the opposite of either handwaving or workarounds.

You could get to it with an FTP or an HTTP client, too. But that would just leave you with the raw files in no particular order, rather than the annotated and organized GOPHER listings. A value-added bonus for the GOPHER version, as I said.

You don't actually have any justification at all for your claim that this is somehow impossible with Debian repositories, given that people like me are doing it with a few simple scripts and even publishing them for the world to see; and clearly neither handwaving nor workarounds for anything are required.

Re:Why does APT not use HTTPS? by notyourday on Wednesday January 23, 2019 @12:37PM (#18980408)
First of all, it is the only thing that is relevant. Repo needs to be functional using tools provided by the distribution that uses that repo format. No one cares that one can download tarballs and compile them -- this is not 1997. In 2019 it is "Enter this command and get this result". Feed this result into the orchestration/management framework. Find a bug? Fix the bug. The rest of the system will continue to function.

Second of all, you still have not provided a link to the doc that someone can read without installing a gopher client. Come on, you said you already have it!

Re:Why does APT not use HTTPS? by kragen on Tuesday January 22, 2019 @01:12AM (#18966328)
> 1. I have installed package X. I want to validate that the files that are listed in a manifest for package X have not changed on a host.

not going to argue with the rest of your flame (others already did) but the answer to this one is

    apt install debsums
Re:Why does APT not use HTTPS? by notyourday on Tuesday January 22, 2019 @08:37AM (#18968110)
This is based around the workflow where one first generates the sums. Notice the -g flag. It is a dumb workaround that I should not need to implement.
Re:Why does APT not use HTTPS? by xcaaa on Monday January 21, 2019 @11:19AM (#18961331)
How is it not trivial if rogue organizations like "Staat der Nederlanden" have default certificates in the browser?

Hint: The Netherlands is world leader in surveillance of its own citizens and inhabitants.

Re:Why does APT not use HTTPS? by kstrauser on Monday January 21, 2019 @12:31PM (#18961887)
TLS instead of GPG signatures might be a bad idea, but adding TLS to the transport of signed packages can't make the pipeline less secure.
Re:Why does APT not use HTTPS? by admax88q on Monday January 21, 2019 @01:45PM (#18962455)
Unless it misleads people in to thinking they don't have to check signatures because they fetched it over HTTPS which is "secure"
Re:Why does APT not use HTTPS? by bigiain on Monday January 21, 2019 @07:08PM (#18964845)
Also, as the article posts out - it's not exactly trivial to deploy https across their global mirror network or to make it work with local caching proxies. That's an easy thing if you've got a handful of servers or a few load balancers, but not so easy or practical for their use case.

(Also, remember most of the apt development had already happened way before free ssl certs became a thing. While saying "Why don't then just use certbot/LetEncrypt is an easy criticism, give them credit for having actually build a GPG sig secured distributed software delivery system years before LetEncrypt existed...)

Re:Why does APT not use HTTPS? by vegardx on Monday January 21, 2019 @11:21AM (#18961349)
Because they would be caught doing it very quickly. There are so many ways to detect this.
Re:Why does APT not use HTTPS? by shittyadmin on Monday January 21, 2019 @12:20PM (#18961796)
And ultimately none of those ways to detect this aren't useful against a sufficiently targeted attack with direct access to the signing key.

It's one thing if you sign a bad key for Google.com, publish it in CT logs and then put it up on the public internet - it's quite another if you sign a bad key for midsizecompany.com, keep it out of the CT system and use it only in a targeted attack against non-technical individuals who are unlikely to examine a certificate or use things like Certificate Watch.

With that said, I still believe serving it over HTTPS would be a substantial improvement. Perhaps pin the cert or at least the CA out of the box to prevent such attacks.

Re:Why does APT not use HTTPS? by Boulth on Monday January 21, 2019 @12:38PM (#18961943)
CT monitors are primarily for site operators not end users. When site operators spot a rouge cert issuance they can clarify that and ultimately get the cert revoked.
Re:Why does APT not use HTTPS? by shittyadmin on Monday January 21, 2019 @12:44PM (#18961983)
Precisely my point - it's not something an end user would notice. And if it doesn't even appear in CT logs or for the end user it would likely go completely unnoticed.

There really aren't "so many ways to detect this" - there's about 3: the user examines the certificate, CT logs catch it later and detection in browsers of major changes on the most high profile sites. Anything falling outside of those will almost certainly go unnoticed.

Re:Why does APT not use HTTPS? by tialaramex on Monday January 21, 2019 @04:54PM (#18963956)
When you (a CA, but also just anybody who finds a new one) log a new certificate or a "pre-certificate" (which is essentially a signed document equivalent to the final certificate but not usable as a certificate) the log gives you a receipt, a Signed Certificate Timestamp, saying it commits to publish a consistent log containing this certificate within a period of time (today 24 hours).

The SCT proves that this particular log saw a signed document with these specific contents, at this specific moment.

Chrome (for a long while now), Safari (announced for early 2019) and Firefox (announced but a bit vague on when) check SCTs for publicly trusted certificates.

The browser can look at the SCT and verify that:

* It was signed by a log this browser trusts

* It matches the contents of the leaf certificate (DNS names, dates, keys, etcetera: Distinguished Encoding means there is only one correct way to write any certificate so there can't be any ambiguity)

* It has an acceptable timestamp (not too old, in some cases not too new)

It can also contemplate the set of SCTs and decide if they meet further criteria e.g. Google requires at least one Google log and at least one non-Google log.

If any of these is wrong, the site doesn't work and an appropriate error message occurs, no user effort is needed or useful here.

We know this works because Google managed to do it to themselves by accident once already, blocking Chrome access to a new Google site for - I think it was several hours - because their dedicated in-house certificate group screwed up and didn't log a new certificate.

There is more to do to defend the system completely:

1. You could compel a log operator to emit an SCT, but then not actually log your certificate. Certificate Transparency can detect this, a browser would need to remember SCTs it has seen and periodically ask a log for a proof which allows it to verify that the log really has included this certificate.

2. You could go further and compel the log to bifurcate, showing some clients a history with your certificate in, and others a parallel history without that certificate. This can only be detected using what is called "gossip" in which observers of the log have a way to discuss their knowledge of the log state and find out systematically if there are inconsistencies.

Once both these things are in place, there's basically no way around just admitting what do you did. Which of course doesn't mean any negative consequences for you, but it does make _deniability_ (much desired by outfits like the NSA and Mossad) hard to achieve if that's something you care about.

Re:Why does APT not use HTTPS? by monocasa on Monday January 21, 2019 @01:36PM (#18962373)
A state actor using a rogue cert is MitMing the connection in a.very targeted way, and the site operator wouldn't ever see it.
Re:Why does APT not use HTTPS? by cb100 on Tuesday January 22, 2019 @08:23AM (#18968040)
This is the most reasonable reply, and of course it is at the bottom. I wonder if accurate technical commentary will ever make it to the top here.