News for hackers. Stuff that matters.
Re:Why Package Signing is not the Holy Grail by IgorPartola on Wednesday July 24, 2013 @08:38AM (#6095792)
The problem is how permissive PyPI is. I can upload a package called django2 with the description that this is the new generation of the Django framework and run arbitrary code on a whole lot of machines of many unsuspecting developers. The reason Linux distributions do not generally have this problem is that there is an air gap between the upstream developer and the repository in the form of a maintainer who reads the code and verifies that it is not malicious. Incidentally, GitHub has the same problem as PyPI. With good SEO I can create a repo called django and have it be the first thing that new developers find.

One possible solution to this is to have a core group of maintainers verify the most popular packages. I place my trust in the maintainer and the maintainer signs a specific version of the package. This will not scale to 100% of the packages, but that is fine. As long as django, flask, psycopg2, etc. are signed I can take responsibility for reviewing django-pretty-snarfs-with-smiles myself. That is essentially our trust model with GitHub already: you blindly download large official looking projects but read through the code of small stuff (right?). Perhaps after a while some developers become trusted and get to have their signing process fast-tracked.

At the end of the day if I could have one improvement it would be a big fat warning that pip gives you that says "what you are about to install is likely malicious code. Do not do it without verifying that it is not by manually downloading it and reading the source."

BTW, simply running "pip install foo" will let foo's setup.py run arbitrary code on your machine. It is such a giant security hole by design that I cringe every time I have to use it.

Edit: another potential model is for PyPI to punt the verification onto outside entities, such as GitHub, Bitbucket, etc. Basically instead off building my packages on my (potentially compromised machine), I would tag a version of it on GitHub and instruct PyPI to build it. It is easy to establish trust between PyPI and GitHub, so the guarantee you get with PyPI is "this package came from exactly this source" where you can read the source before installing it. Like I said above, GitHub also has a trust model problem but this would reduce the amount of work one would need to do to verify a package from two places to one.

Re:Why Package Signing is not the Holy Grail by pekk on Wednesday July 24, 2013 @10:49AM (#6096473)
Suppose that 'pip install foo' did not run a setup.py. Now eventually you are going to run some code from foo. If you weren't, why are you installing it? At that time, you could have 'arbitrary code' running on your machine.
Re:Why Package Signing is not the Holy Grail by IgorPartola on Wednesday July 24, 2013 @04:06PM (#6098747)
My point is that you don't even get a chance to review the source before it runs. pip runs in trusted code automatically, without any verification whatsoever. At least when you run 'git clone' that does not happen.
Re:Why Package Signing is not the Holy Grail by donaldstufft on Wednesday July 24, 2013 @05:57PM (#6099384)
The recently released pip 1.4 allows you to install from Wheels which does not use setup.py during install and executes no code from the package during install.
Re:Why Package Signing is not the Holy Grail by IgorPartola on Thursday July 25, 2013 @08:24AM (#6102039)
Yes, except I have seen very few packages that are not sdist's. Another issue is that unless the package maintainer disables it explicitly on PyPI, pip will go searching over the package's home page for a newer version. This is just broken since even if PyPI was to implement some type of package verification, example.com/~/devfoo/ might not. Often times these pages do not even have HTTPS set up.

As you said, this is a complex problem, but I think the first step is to take care of the easy things. Let me know if you'd like to chat off HN about this.