Key X is on the company approved key list, key y is not. Your argument just fell apart.
>The problem with manual review then becomes one scale. There are as of this time of writing 117,226 unique projects on PyPI with an average growth of around 100 new projects a day.
You're not dealing with projects. You're dealing with keys. It's not one key per project. It's one key per contributor. This has the added bonus that if a contributor goes rogue, you can revoke the one key and all the suspect projects are are invalidated at once.
>There is one part time paid person (me), plus my unpaid time, plus one other part time unpaid developer/ops person who do the vast bulk of the work.
Sonatype has turned this into a rather nice business. It's not a volunteer project for them. You expect me to believe it's impossible despite solid examples to the contrary?
>at the end of the day without a manual review system individual end users are still ultimately responsible for ensuring they're asking for the correct thing
Blaming the victims.
>Security is achieved by layering multiple secure systems on top of each other, not by randomly rubbing crypto on things because it makes you feel good to have crypto involved.
It's also not achieved by doing absolutely nothing at all.
My rough guess is that for the Python community, these are roughly proportional; there are a lot of different people maintaining approximately one library each, not a small number of people (or companies) maintaining large parts of the ecosystem. There's nothing directly like org.apache for Python.
I think the only way your key-signing mechanism would actually solve the problem is if we made it actively hard for new developers to upload projects to PyPI without a long vetting process. Some projects work this way (Debian, notably; I've had upload rights for a few Debian packages for years and still don't feel ready to apply for full access), but I think it's a poor match for PyPI's actual goal.
You know who does this sort of thing? Politicians. They can't just look at a working system, single payer for instance, and copy it. No, they have to make silly arguments about why it will never work, despite a concrete, working example, right in front of their own eyes.
If Python introduced manual review of new packages, it would either need a massive amount of resources that no-one is offering to provide, or it would immediately be a huge bottleneck on people making new packages, which the community doesn't want to do.
Lipstick on the pig, still covered in mud.
The key difference is the regular occurrence of malware finding its way into PyPi and NPM due to the lack of multilayered security on those repos.
You guys keep trying to prop up the straw man that ONLY package signing is needed. It's not. It's a start. Nobody is making that argument but you. You not only repeatedly beat that dead horse, but you carry it to the illogical extreme that package signing is somehow harmful. Not only do you see no value in that layer of security, but you actively resist any talk or attempts at implementing it.
Meanwhile, your repo is infested with hackers and malware. Big surprise.
you cannot locate said rogue contributor without regularly manually reviewing 117,226 packages.
Herd immunity. Someone is out there reviewing it. Most users won't need to lift a finger beyond verifying signatures.
More likely everyone assumes someone else is reviewing it, and nobody actually does.
A minuscule amount of people are going to bother to do something like approve keys. Security for the minority can already be achieved by those companies mandating their developers use DevPI and mirroring trusted projects from PyPI to DevPI (or similar system).
Complicating the system further for something that, for practical purposes, does not improve the security of the vast bulk of people is not a trade off we're willing to make. Package signing will come to PyPI, likely in the form of TUF which is strictly superior to the trust model provided by PGP for package signing. It hasn't done so because nobody has had the time to do it yet.
What you seem to be missing about my statement both in blog post and here is not that package signing is not worthwhile, but that a lot of people like yourself seem to think that all you need to do is add signatures to a system and suddenly poof it's secure! That view point is common among inexperienced developers or people who don't commonly think too hard about how secure systems are designed/made.
The reality of the situation that adding signatures is painfully easy, but that without a coherent trust model backing those signatures you've achieved nothing but adding more complexity. Determining a trust model (particularly one that works for the majority) is the hard part, and you can't just wave your hand and wish it better.
> Sonatype has turned this into a rather nice business. It's not a volunteer project for them. You expect me to believe it's impossible despite solid examples to the contrary?
Is it impossible to turn PyPI into a business? I don't suspect it is no. However I don't want to do that because my personal risk tolerance doesn't have room for giving up a stable job with health benefits for something that may or may not fail. Others are free to try that if they want of course, but given the lack of people stepping forward to do that, it doesn't seem like anyone else is interested either.
> Blaming the victims.
Stating reality. PyPI is not a curated repository and the end users is responsible for their own security while using it. If they wish to outsource that responsibility there are a number of Linux distributions that are happy to do that for them as well as companies like Enthought and Continuum Analytics who provide curated repositories.
> It's also not achieved by doing absolutely nothing at all.
Good thing we're not doing nothing at all then. Luckily for the Python community we have actual experts and not arm chair cryptographers who fail to understand even the basic fundamentals of developing secure software.
This is the weakest argument. Are Python devs somehow dumber than Java devs? Are they dumber than Android devs? Are they dumber than iOS devs? Everyone knows how to sign a dependency/app/project except python devs? I don't believe that. I honestly think that's the most insulting aspect of this argument.
The rest of this post seems to have turned to hand waving and personal attacks, so I won't bother responding to that. I'm just glad I got to share this perspective with you. Once you cool down, I hope you look harder at the problem. All I care about is improved security. I'm not here for the imaginary internet points.
Nope, I think they're perfectly capable of signing things. I also think it's silly to ask them to do that when the proposed system hasn't been designed to provide any benefit. Properly designing that system is hard, and 99% of people who go "just use PGP!" or "just use X" have spent exactly zero amount of time doing that. Particularly when the proposed solution doesn't actually solve the problem at hand (though it does solve other problems if it's correctly designed).
Ultimately your "suggestions" are nothing new, they're the same generic, cargo culting, suggestions that folks who haven't looked really hard at the problem tend to make.
Maven Central, to my knowledge, does not have typo squatting problems because Sonatype has a manual review process for all new projects. It has absolutely nothing to do with the fact that they allow projects to upload PGP signatures and it could not have anything to do with that, because PGP does not provide any mechanism to prevent that.
For example, there may be `urllib3` which is a valid project that must be signed by key X. We'll ignore how a tool like pip would find out that key X is the right key (although this is actually the most important part of a package signing solution) and just grant that we've solved that problem. Someone then comes and registers another project, `urlib3` which must be signed by key Y. The attack that is being described here is that a user would erroneously say ``pip install urlib3`` when they meant to type ``pip install urllib3`` and pip would then fetch that and download the package and install it. I think it is pretty obvious that signing doesn't help here, because pip doesn't know that the user really wanted urllib3 and not urlib3, so it can only determine that urlib3 is supposed to be signed by key Y (which of course, the hypothetical malicious person controlling urlib3 would have), fetch the package and verify it's signature.
There is only one tried and true method for preventing across the board this kind of human introduced error collision (aka typo squatting), and that is manual review of all new projects. The problem with manual review then becomes one scale. There are as of this time of writing 117,226 unique projects on PyPI with an average growth of around 100 new projects a day. In addition there are zero full time developers or operations or support people working on PyPI. There is one part time paid person (me), plus my unpaid time, plus one other part time unpaid developer/ops person who do the vast bulk of the work. There is simply not enough available bandwidth to process 100 new projects every day and to validate them for typo squatting/confusion possibilities.
Beyond that, there are a number of possible heuristic based approaches that can try to reduce the chance of this from happening such as using levinstein distance, unicode confusables, attempting develop "reputation", etc. Most of these are either so broad as to catch a lot of projects which are not typo squatting but are real, actual different things or are so narrow as to be trivially defeated. That's not to say they aren't worthwhile or there isn't an idea that would make sense but focusing on that has not been a priority for a largely volunteer based organization because there are lower hanging fruit that are more impactful , because at the end of the day without a manual review system individual end users are still ultimately responsible for ensuring they're asking for the correct thing (and even beyond that, they're responsible for ensuring that the thing they're asking be installed is something that satisfies their own security constraints).
Security is achieved by layering multiple secure systems on top of each other, not by randomly rubbing crypto on things because it makes you feel good to have crypto involved.