News for hackers. Stuff that matters.
Re:Linus' reply on Git and SHA-1 collision by orblivion on Friday February 24, 2017 @02:16AM (#13721710)
I control a fleet of servers. I have a saltstack or ansible script. One of the steps in provisioning a new server is to pull library X from github.com.

One day Egor Homakov finds a new hack and finds his way into access to the master branch for library X. As a prank, he force pushes a change to master.

Being aware of such a possibility, instead of setting up my script to pull from master or even a specific tag, perhaps I pull from a specific sha1sum. That way I know I'm getting the exact version I want. That's today. Tomorrow, when preimage SHA-1 attacks are cheap, it will no longer save me.

Re:Linus' reply on Git and SHA-1 collision by stonogo on Friday February 24, 2017 @02:43AM (#13721806)
Sounds like sentence three is where you got it wrong. You should be pulling your software to a development host, building it after verification, and uploading the deployment package to an artifact repository inside your security perimeter. Sentence three becomes 'pull library X from my artifact server.'

There is no way to securely deploy a package directly from the internet. The sooner you understand that, the sooner you'll be able to sleep through needless catastrophes like this hypothetical attack -- or leftpad.js.

Re:Linus' reply on Git and SHA-1 collision by Perseids on Friday February 24, 2017 @04:04AM (#13722164)
> There is no way to securely deploy a package directly from the internet.

Fetching a file from the internet and verifying it against a hash or signature is the only way to securely deploy a package from the internet. This is exactly one of the use cases theses cryptographic primitives are built for. Don't blame the users when they utilize the tools the way they are supposed to be, blame the tools when they fail when doing so.

And by the way, if you ever need to run `apt-get upgrade` or `apt-get install` on your production server from a public mirror, then you are guilty of "deploying a package directly from the internet", too. (My apologies and congratulations if don't.)

Re:Linus' reply on Git and SHA-1 collision by raverbashing on Friday February 24, 2017 @04:50AM (#13722340)
apt-get can (and do) check signatures of repos

But yeah, you could put an unverified repos to be used

Re:Linus' reply on Git and SHA-1 collision by Perseids on Friday February 24, 2017 @05:47AM (#13722565)
For the argument at hand (Can you deploy anything on a production server from a third party that is only verified cryptographically?) signatures and hashes fulfill the same function (and btw I also wrote "verifying it against a hash or signature" above). More to the point, under the hood GPG signatures only sign a hash of the file in question anyway. Verifying a file via a GPG signature is strictly less secure than verifying it by its hash (assuming you use the same hash function as the signature).
Re:Linus' reply on Git and SHA-1 collision by lmm on Friday February 24, 2017 @05:02AM (#13722406)
Git has tag signing, surely there's a way to clone a specific tag and check the signature against a specific GPG key fingerprint?
Re:Linus' reply on Git and SHA-1 collision by nhaehnle on Friday February 24, 2017 @06:22AM (#13722689)
Tag signatures only cover the mapping from tag name to commit hash. In other words, specifying a manually-verified commit hash is actually more secure.

Tag signatures are mostly worthless now from a crypto point of view -- with the caveat that you can still get some value from them if you still trust sha1 to be secure against second-preimage attacks.

Re:Linus' reply on Git and SHA-1 collision by heartsucker on Friday February 24, 2017 @06:19AM (#13722678)

    git checkout 0.1.0
    git tag -v 0.1.0
Re:Linus' reply on Git and SHA-1 collision by amingilani on Friday February 24, 2017 @04:01AM (#13722153)
Either way he'd be pulling from Github.com at some point, whether in the build environment or production. He won't ever know that the repository was compromised.

Besides, some people use Heroku simply to deploy their rails or node apps, and they don't have an artefacts server.

Re:Linus' reply on Git and SHA-1 collision by dozzie on Friday February 24, 2017 @03:23AM (#13721999)
> I control a fleet of servers. I have a saltstack or ansible script. One of the steps in provisioning a new server is to pull library X from github.com.

Ansible gives larger and easier to exploit attack vectors than git.

And then, with pulling semi-random things from internets you have much bigger problems with your deployment procedure. You should never ever download a git repository with software, instead you should be using package system supplied by your operating system.

Re:Linus' reply on Git and SHA-1 collision by mverwijs on Friday February 24, 2017 @04:00AM (#13722150)
> you should be using package system supplied by your operating system

2005-me vehemently agrees with you.

Re:Linus' reply on Git and SHA-1 collision by amingilani on Friday February 24, 2017 @04:02AM (#13722157)
Package managers allow you to download from git repositories too. He'd suffer from the same attack vector.
Re:Linus' reply on Git and SHA-1 collision by dozzie on Friday February 24, 2017 @04:04AM (#13722162)
Erm... Which ones?
Re:Linus' reply on Git and SHA-1 collision by shasheene on Friday February 24, 2017 @04:15AM (#13722209)
I believe Homebrew on macOS uses it for formulae (Ruby scripts to actually download and build the application)
Re:Linus' reply on Git and SHA-1 collision by dozzie on Friday February 24, 2017 @04:26AM (#13722249)
Well, I don't hear too often about a fleet of servers running MacOS (because this branch of comments started with provisioning in a fleet of servers).

Also, from rants I hear about Homebrew and its breaking random libraries on upgrades, I don't think it should be mentioned as it was state-of-the-art or something.

Re:Linus' reply on Git and SHA-1 collision by amingilani on Friday February 24, 2017 @05:30AM (#13722506)
Re:Linus' reply on Git and SHA-1 collision by dozzie on Friday February 24, 2017 @06:32AM (#13722723)
And which of these are "package systems supplied by your operating system"?
Re:Linus' reply on Git and SHA-1 collision by sqeaky on Friday February 24, 2017 @02:52PM (#13727135)
While I agree that the OS package should be used first and foremost, it just often doesn't have the required software. (Even after adding extra repos that it might support)

You are picking on the useless details. Those are all commonly used package managers in production environments. They often provide software that simply never gets packaged with the OS. They likely always will, because they have more focused design goals.

A better way to argue this would be point out that specific ways to use the package managers better. For example: bundlr supports saving all the required packages offline. This provides the opportunity to do a security review and save the packages locally/internally rather than always trusting the whatever is on the Internet.

Re:Linus' reply on Git and SHA-1 collision by dozzie on Friday February 24, 2017 @03:10PM (#13727372)
> While I agree that the OS package should be used first and foremost, it just often doesn't have the required software. (Even after adding extra repos that it might support)

OS-supplied packages don't grow on magical trees. If you don't have the necessary software in official repositories (or if it's your software), you can package it yourself. Deployment then becomes a breeze, and you save yourself otherwise completely useless process of recompiling things over and over again.

> You are picking on the useless details.

Quite the contrary. Those details make important difference.

> Those are all commonly used package managers in production environments. They often provide software that simply never gets packaged with the OS.

Apart from Homebrew, which is for workstations (hardly anybody runs macOS servers), none of these "package managers used in production environments" provide you a complete way to rebuild your software. You can be fine for a while if you stay away from modules that are interfaces to C or C++ libraries and from tools from other languages (e.g. I have used Python's Sphinx to document Erlang daemons quite successfully), but once you hit that, deployment starts to be PITA, because you'll need to remember to install all the required libraries, -dev packages, compilers, and what not.

On the other hand, DEB or RPM with artifacts will just automatically pull the required libraries, and its build dependencies give a dedicated and standard place for the necessary build tools.

Your comment supports my opinion that today's programmers usually don't want to be bothered with learning things that have been working for sysadmins for twenty years already.

Re:Linus' reply on Git and SHA-1 collision by Xylakant on Friday February 24, 2017 @03:17PM (#13727473)
Gentoos emerge and FreeBSD port are package managers that preferably build from source. At least emerge also accepts git sources, with tags and commit specifiers. I haven't used bsd in a while on a prod system, but I'd not be surprised if port gained the same functionality - after all whether you're pulling a tarball and use a checksum for integrity checking or hand the task off to git makes exactly no difference at all.

And I have yet to hear that port and emerge are incapable of doing dependency resolution. They're battle tested systems that work well in production environments.

Re:Linus' reply on Git and SHA-1 collision by dozzie on Friday February 24, 2017 @03:35PM (#13727643)
First, portage and ports are OS-supplied mechanism for installing software. They are nothing like pip or gems or npm, which only can install things written in their respective languages of choice and fail miserably for modules touching any library external to them (unless you manually ensure the library's and compiler toolchain's presence, that is).

Second, ports and portage have support for and networks of mirror servers that keep copies of software available through these packaging systems. It's trivial to switch if one of the mirrors goes down. For pip, gems, or npm you need to plan ahead for the problems and deploy your own package cache, from what I know.

Third, I was using Gentoo with one of these "battle tested systems that work well in production" for several years. It was doable, but it wasn't pretty, could lead to breaking software after updating some random deep dependency (if it was recompiled with different flags), and generally required more work and attention than APT would, all that for very little gain (if any gain at all). Oh, and it ended up working with binary packages, after all, I just needed to compile them myself instead of having a half an hour downtime of production MySQL because it needed to get compiled (which could fail, leaving me with no working database installed).

Re:Linus' reply on Git and SHA-1 collision by sqeaky on Friday February 24, 2017 @04:17PM (#13727978)
Gems can be used to installing anything. It often build java, go, C, C++ and I think I heard it could do rust once, but generally anything in faster languages to create faster versions of the implementation of the functionality for a given gem.

I bring that up to highlight some of the wrong assumptions you make. You make several needless assumptions and use those to draw funny distinctions between things. I am not even sure of the point anymore.

likely any of these systems could be used in a variety of environments for a variety of purposes.

Re:Linus' reply on Git and SHA-1 collision by steveklabnik on Friday February 24, 2017 @05:18PM (#13728456)
You can do anything, including Rust, yeah. You can also distribute pre compiled stuff.
Re:Linus' reply on Git and SHA-1 collision by Florin_Andrei on Friday February 24, 2017 @07:02PM (#13729110)
The ones that actually matter are more like apt or yum.
Re:Linus' reply on Git and SHA-1 collision by yellowapple on Friday February 24, 2017 @06:05AM (#13722617)
Most language-specific ones do. OS-specific ones less commonly do; Chocolatey, for example, can (IIRC) install packages by running PowerShell scripts, which may or may not include pulling from GitHub.
Re:Linus' reply on Git and SHA-1 collision by dozzie on Friday February 24, 2017 @06:34AM (#13722729)
> Most language-specific ones do.

Those are development tools, not deployment ones (even though they are used as such; programmers usually don't bother with learning what sysadmins do, so it's not a surprise).

Re:Linus' reply on Git and SHA-1 collision by Xylakant on Friday February 24, 2017 @06:58AM (#13722812)
getoos emerge can. A package definition contains a source URL and git urls (with branches and tags) are accepted sources.
Re:Linus' reply on Git and SHA-1 collision by Faaak on Friday February 24, 2017 @04:53AM (#13722351)
yaourt on archlinux
Re:Linus' reply on Git and SHA-1 collision by hprotagonist on Friday February 24, 2017 @09:49AM (#13724090)
Pip, too.
Re:Linus' reply on Git and SHA-1 collision by danmaz74 on Friday February 24, 2017 @07:08AM (#13722848)
> Tomorrow, when preimage SHA-1 attacks are cheap, it will no longer save me.

Well, Moore's law is petering out isn't it? :)

Re:Linus' reply on Git and SHA-1 collision by runholm on Friday February 24, 2017 @09:34AM (#13723940)
Computers are not getting better at the rate they used to, but we are still improving both on performance and power.

However, the largest change we have seen the last couple of years is the accessibility of computer power. These days an attacker can be concerned mostly by how many CPU/GPU hours he needs to rent from Amazon or equivalent providers to achieve his goal. So the accessibility, convenience, and price of processing power is still improving quite fast.

Re:Linus' reply on Git and SHA-1 collision by sqeaky on Friday February 24, 2017 @02:35PM (#13726934)
Depending on your interpretation. If you go by the relatively useless transistor/fixed price cpu then it is dead, but if you go by more practical measures like instructions per second in whole system/dollar the it is as fast as ever. GPUs are really fast and still advancing.
Re:Linus' reply on Git and SHA-1 collision by officelineback on Friday February 24, 2017 @09:49AM (#13724098)
Moore's law is, but the cost of a given amount of computational ability is still going down quickly.
Re:Linus' reply on Git and SHA-1 collision by hasenj on Friday February 24, 2017 @03:13AM (#13721957)
If you want a specific hash, why pull it from github everytime?
Re:Linus' reply on Git and SHA-1 collision by falsedan on Friday February 24, 2017 @03:16AM (#13721973)
Because, when provisioning a new server, where else can you get it from?
Re:Linus' reply on Git and SHA-1 collision by wjh_ on Friday February 24, 2017 @03:21AM (#13721993)
A host that you control.
Re:Linus' reply on Git and SHA-1 collision by falsedan on Friday February 24, 2017 @09:12AM (#13723772)
Where did it get it from?
Re:Linus' reply on Git and SHA-1 collision by hasenj on Friday February 24, 2017 @10:23AM (#13724416)
From github, the first time only. Not every single time.
Re:Linus' reply on Git and SHA-1 collision by falsedan on Saturday February 25, 2017 @05:19AM (#13730940)
Isn't getting it from GitHub the first time vulnerable to attack? Consider when you're scaling up or replacing a failed host.
Re:Linus' reply on Git and SHA-1 collision by dozzie on Friday February 24, 2017 @03:20AM (#13721990)
From a custom package you have built? Did you know that RPMs and DEBs don't grow on magical trees, but are built?
Re:Linus' reply on Git and SHA-1 collision by throwayguestman on Friday February 24, 2017 @04:48AM (#13722331)
So, the problem still stands, theoretically.

Not only do you likely need to populate your host with packages not from your host. But also, your host will also still be connect to a public net, even if only indirectly (e.g. private net), and hence potentially manipulated.

Re:Linus' reply on Git and SHA-1 collision by dozzie on Friday February 24, 2017 @04:57AM (#13722381)
No, you misunderstood what is actually the problem here. Pulling under deployment some code from random resource from the internets that can go down or get deleted at a whim and you can't easily move to just some other mirror and you don't even control when the thing will be up back, that's the problem. Not the trust you need to put to use the code (this is still there, obviously). And the very same comment applies to third-party package repositories, like PPAs in Ubuntu.

Not to mention that with pre-built binary packages your deployment speed and repeatability get significantly better, as you don't need to rebuild the artifacts every single time.

Re:Linus' reply on Git and SHA-1 collision by throwayguestman on Friday February 24, 2017 @05:30AM (#13722509)
I do understand, but "the trust you need to put to use the code" is what we were talking about, no?