Each time I'm about to run `curl https://install.newhotness.io|sudo sh` I'm left with the following questions:
* Will it work? I'm on Arch Linux. This installer was probably tested on OSX and Ubuntu and deemed "portable". I've seen curl|sudo sh installers trying to apt-get installing dependencies.
* Where does it place files? I've got a ~/.meteor folder that's 784MB. Why would you install software libraries and binaries to $HOME? I now have to tell my backup tool to ignore it when backing up my home partition, great. The FHS was established for a reason, people.
* Correlary to the previous question: How do I uninstall it? Maybe all files were installed to a hidden folder in ~/, maybe there's some stuff in /opt, or /usr/local, or who knows where else. If the installer doesn't implement uninstall functionality itself, I now have to go hunt for the stuff it placed in my filesystem.
I understand the need to be able to distribute your software without having to implement native packaging for OSX/Debian/Ubuntu/Red Hat/Fedora/Arch/...
Docker solved this problem. If you're feeling lazy when it comes to distributing your app, just ship a Docker container and be done with it.
How does one uninstall stuff on Linux anyway? My Linux boxes are always like thrash cans - I add and add new stuff until it's time to upgrade either the system or the machine, so it gets wiped and replaced. Never figured out how to delete stuff in a way that will not leave a ton of things behind...
sudo apt-get purge --auto-remove packagename
with a package manager suited for your system, or with the knowledge of what files the software in question touches.
The packages authored by distributions tend to be pretty good, but I've seen RPMs with nothing more than tar zxf in them and no tracking of the files installed.
The major package managers expect the package to manage the artifacts installed, this works if everyone plays nice and does their job...
The manager basically just defines a framework. Packages can abuse that framework in various ways (I haven't even mentioned install triggers, to execute code when other named packages are added or removed). There are a lot of ways to get it wrong, but when the dev builds a good package, it's almost magic how nicely it works.
$ touch now
$ install.sh
$ find / -newer now
You can redirect the output of find to a file for later removal. You will have to filter out false positives from the 'find' output, usually dev nodes.
The closest to cruft is hidden directories in my homedir from software I don't use anymore. That isn't a problem either: they don't take up much space, and it's easy to manually delete them if I ever feel like it. If not, they are compressed and deduplicated by backup software, and they're hidden by default, so who cares?
A lot of people seeking and offering help on forums are former Windows users, who are used to doing that to fix problems. They just don't what else to do, so that's what they do.
And, sure, if you break a configuration file or package dependency, wiping the disk and reinstalling will fix that--but so would fixing the file or dependency. In my experience, the nuclear option is never necessary with Linux. Worst case, you boot off a CD or USB drive, mount the partition, chroot if necessary, fix it, and reboot.
> Other times it makes things worse.
That is very strange, indeed. D:
> buying a lottery ticket soon?
Haha, nope, been having this "luck" for years now, but it only seems to affect my Linux installs. :(
And just yesterday my Linux did what I've never seen an OS do before - it losts it UI clock display. The solution was simple and typical for Linux - it involved killing a random process. But at that moment I realized that we used to laugh that rebooting stuff is the Windows way of doing things. Not anymore, apparently.
;).
What desktop environment was that? There are many different ones available for Linux systems, and they don't all behave that way. e.g. I've never had that kind of problem with KDE3/TDE or KDE4.
That is called sedimentary data storage, ie. piling new files on top of the old ones.
Every time I've seen it used, curl | bash is simply a way to deal with the fact that the users all have different package managers (often with outdated software) and quite complicated workflows to get stuff set up with them because there were problems that the package manager didn't solve.
>Docker solved this problem.
Ironic. Docker is one of those applications. And they used to ask for it to be installed with a curl pipe.
They seem to have replaced it with what is effectively a script executed with human hands. Not really an improvement.
There's a section on their website[0] there are installation instructions for various distributions using repositories and package managers. In other words, the right(tm) way of doing software distribution on Linux.
But then on a different part of their website I found the curl | sh instructions you allude to[1]. Bizarre that they still support that installation method.
To make matters worse, the big orange "Get Started" call to action button actually leads to the curl|sh installation instructions. In light of that I certainly can't blame you for not knowing about the proper distribution packages.
Point being that even if you install "the right way" on linux using package managers it's still a multi-step complicated process that benefits from being automated in a bash script.
It's really a shame that you have to wrap the best practices aproach in a bad practice in order to make it convenient.
> Point being that even if you install "the right way" on linux using package managers it's still a multi-step complicated process that benefits from being automated in a bash script.
Right, because distributions want different things than software authors.
If you're a Debian/Ubuntu/RHEL maintainer, you want stable, tested versions of software that you can vouch for and support for several years.
If you're the maintainer of an application, in this case Docker, you want your users to run the latest and greatest version of your software. You can't just demand that each distro just ships the latest and greatest version of your software. So if you're Docker you end up having to do the following for each distro:
* package the software for that distro
* sign that package
* serve your your own repository containing that package
* ask your users to add the repository
* ask your users to trust the signature
That's one of the reasons I like running Arch Linux on my workstation. The desires of the software authors, distro maintainers and users align.
Docker Inc: "Hey wouldn't it be cool if your users could install the latest
stable version of Docker?"
Arch Linux maintainers: "Yeah, it would. We'll keep the latest version in the
repo and our users can just `sudo pacman -S docker`"
Docker Inc: "Cool."
Arch Linux users: "Cool."You rarely see curl|sh installers for server stuff though. I think the topic at hand mostly concerns workstations.
# This is the script that we install somewhere in your $PATH (as "meteor")
# when you run
# $ curl https://install.meteor.com/ | sh
# It's the only file that we install globally on your system; each user of
# Meteor gets their own personal package and tools repository, called the
# warehouse (or, for 0.9.0 and newer, the "tropohouse"), in ~/.meteor/. This
# means that a user can share packages among multiple apps and automatically
# update to new releases without having to have permissions to write them to
# anywhere global.A few points of the shortcomings of curl|bash vs RPM and others:
* curl|bash is not transactional. If you have an install/upgrade process in a different tab, and you close your terminal accidentally, how do you know if the install/upgrade completed?
* There's no obvious uninstall method, and you need to read the docs to see what, if anything, you should remove outside the main directory.
* There's no way of verifying if files have been tampered with post installation. This has negative security and operational consequences.
* It's much more vulnerable to MiTM attacks - a proxy with a successfully faked certificate can trivially modify a bash script on the wire and add a malicious command to it. This is hard for a user to detect. Packages are (on most distros) installed via package managers that verify GPG signing keys, making in-flight modification very very greatly more difficult.
* Packages are much more auditable - I can download a package and inspect it, and know that it will run the same actions on every machine it's deployed to. curl|bash can trivially serve different instructions to different requests, based on whatever secret criteria the server operator decides.
If you're going to try and argue that instead of doing what the generally accepted $good_thing is, and instead use $quick_and_easy solution that's widely perceived to be insecure, when you argue in defence of your position, you need to make very sure that you're not just trying to justify laziness. I don't think sandstorm have really thought about this hard enough.
1. "curl|bash" vs "curl >install.sh; sh install.sh" 2. install scripts vs package managers
The OP talks about how to detect a streaming execution of a script and exploit it. This proof of concept exploit aims at discouraging the the use of curl|bash as a means to installing code, and instead promotes that at least you should save the script (and review it) (i.e. the "curl >install.sh; sh install.sh").
All of the points you mentioned are valid and important pain points that affect any arbitrary executable install script (not verified cryptographically using a side channel). Deciding whether to use an install script or a packaging system is really a tradeoff.
The "obvious" problem the OP was talking about was that smell you feel when you see piping code to bash. I think OP made an excellent job showing how you can make it harder for people to spot and audit malicious code and how this is pretty serious downside for piping code into bash; It's more like: "I have a gut feeling this is horrible, so let's think about a way to prove that gut feeling", and the proof is insightful and non obvious.
Perhaps they should have said: "however, the following non-security arguments are valid, and there are more we didn't list." Agreed.
But, at the end of the day, they're right: it's not _less secure_ than, e.g., npm.
PS: In your list; "It's much more vulnerable to MiTM attacks" --- that's essentially the PGP vs HTTPS argument in disguise. Which they also cover, and acknowledge to be true; the argument is essentially "curl https:.. | bash is not less secure than any HTTPS based method, including npm, .isos, etc."
EDIT: and because this is such a hot topic, I just want to emphasize: I'm not arguing in favor of curl | bash. This is only a counter-argument to "curl | bash is less secure than other https- based install methods".
My point with the PGP argument is that you can indeed go through all those steps to verify their install script with GPG (although note that on top of the steps they run, you should ideally also verify the script against a known hash that you verified elsewhere to make sure that the contents of the script is what you expect, as well as the authors).
But the manual GPG verification process will be ignored by almost all users, who will decide they can't be bothered. My point about package managers is that they do this automatically, without requiring the user to jump through extra hoops. Which is much better than the 8 additional steps that almost no-one will bother with, in practice.
Also saying that it's not less secure than npm isn't a great benchmark - npm is really bad too. See the recent left-pad issues, for example. So yeah - it's not really much worse than other things which are also bad. But it's a lot worse than doing it properly, with a signed RPM repository.
If you've verified the script with GPG, the extra step of a hash is wholly unnecessary. Any change to the script's contents would immediately invalidate the GPG signature.
The point of this oneliners is to allow anybody to install it. It's insecure as hell, but whether you're piping it directly to bash or downloading an installer file and then executing it (like a great deal if not most of the software is still distributed to the general public anyway) makes not a big difference.
That's why the linked article is interesting; it shows that in fact there is a difference between piping to bash or downloading an installer. The difference is important and concerns the auditability. Any people who cares to audit the script might not notice anything suspicious but people who blindly pipe it to bash can potentially run code that is not the same that audtors see.
What's basically happening is the either a malicious software publisher, or a MiTM, fools auditors. And fooling auditors is bad because a lot of people rely on the community to signal this kind of abuse.
What an auditor could do to detect this kind of malware distribution is:
curl https://foo/bar.sh | tee >(md5sum)| sh
However, people who follow good practices and want to audit the script are not affected by the problem in the first place, because they will just save the script and they will know how to execute it after having read it.
People who don't know how to audit the installer in the first place might not even know how to verify the download with GPG as well. Or worse, they won't bother and might deem the software to be too hard to install and choose to use something worse.
That's why sandstorm offers PGP-verified installs for those who know what they're doing and show how to use keybase to solve the web of trust issue.
(I'm not affiliated with sandstorm, but they seem a very reasonable and competent bunch to me)
I suspect we've spent far more time thinking about this than most people. We've heard every argument many times. The referenced blog post specifically addresses the security argument (hence the title) because that's the one we feel is most misleading.
I absolutely agree that curl|bash is ugly compared to package managers, but there are some serious shortcomings of package managers too, and it's a trade-off. With deb or rpm, we are tied to the release schedules of the distros, which is unreasonably slow -- Sandstorm does a release every week whereas most distros don't release any more often than twice a year.
We can perhaps convince people to add our own servers as an apt source or whatever, but now the MITM attack possibility is back. Yes, packages are signed, but you'd have to get the signing key from us, probably over HTTPS. No one wants to do the web of trust thing in our experience (I know because we actually offer this option, as described in the blog post, and sadly, almost no one takes it).
The Sandstorm installer actually goes to great lengths to be portable and avoid creating a mess. It self-containerizes under /opt/sandstorm and doesn't touch anything else on your system. This actually means that Sandstorm is likely to work on every distro (given a new enough kernel, which the installer checks for). If we instead tried to fit into package managers, we'd need to maintain half a dozen different package formats and actually test that they install and work on a dozen distros -- it would get pretty hard to do that for every weekly release. (Incidentally, Sandstorm auto-updates, and it does check signatures when downloading updates.)
So, yeah, it's pretty complicated, and we're constantly re-evaluating whether our current solution is really the best answer, but for now it seems like what we have is the best trade-off.
Note that distributing Sandstorm as .debs or .rpms wouldn't solve this, because we'd still be distributing them from our own server, and you'd still need to get our key from somewhere to check the signature.
$ ./configure
$ make
sudo make installEither way I'm not saying `curl | bash` is good, I'm just trying to point out that people have a very knee jerk reaction to it and pay no mind to things which are similarly as dangerous.
[1] https://l3net.wordpress.com/projects/firejail/ [2] https://github.com/teran-mckinney/raru
It's not like your user password is 1234, right?
https://www.gnu.org/prep/standards/html_node/Standard-Target...
The idea is that if one doesn't trust a random curl-ed install file, they shouldn't trust a downloaded makefile either -- and for the same reasons, yet tons of people complaint about the first, but have been using the latter for decades without complain...
It's not about where it will install it if it works fine.
Install software from verified sources.
Whether you use curl or not is irrelevant.
This is what the Debian guys standing in a circle holding their passwords are doing at your Linux event. It's also why when you visit github it says 'GitHub, Inc [US]' in your browser's address bar. Heads up: I work on the latter.
There is, I'm sure, malicious software hosted on github, so I can download things from that trusted source and still have trouble.
there is insecure software hosted on github, again there's no protection based on hosting company.
If a developer who hosts a repository on github has their credentials compromised, their software may become malicious.
Also as an end-user (particularly a non-paying one) I have no visibility of Githubs own security policies and practices, so assessing the level of trust placed there is tricky.
The debian model has more checks and balances (i.e. it's harder to get from one set of compromised creds to a malicious package in a production repo.) but still not perfect...
Passports, surely? :P
If something is from e.g. Red Hat or a large project repository like Apache, it's more trusted than a random website with some unknown tarball...
If many people in websites and forums say "don't trust this program", well, don't trust it.
Etc.
On the internet, how do you know who's who. you download a package from npm. Who wrote it, what's their background, where do they work? What are their security practices like, what are the security practices of the hosting company?
so we say Redhat/Apache are more trusted sure, that's one of the reasons that I dislike curl|bash from random sites, there's no way of assessing trust in that software or its origin...
Check the npm package's ratings, and the github repository watchers and stars.
Try to find and read a few posts (and their comments) on the package. The more respected the source (e.g. HN comments vs comments on Digg), the better.
Even there though all it does is move the problem one step away. Most npm packages have dependencies (and quite a few of them), you have no way of knowing whether the author of the main package did the checks you describe for their dependencies...
That's the nice thing: you don't need to. You just need to trust that the main package works well enough -- and you get that from the fact that it is in widespread use (many downloads, watchers, stars, posts, etc).
the fact that no-one else is checking and you are therefore in the same boat as other compromised people might not be that much of a consolation
Beyond that, yeah, nothing is absolutely certain in life except death and taxes...
For example OpenSSL was and is used by lots of people, this has not stopped it having a lot of security problems...
And true once the vulnerabilities get published you get to know about the problems more quickly but then in a world of targeted attacks and professional exploitation that might not be great comfort..
The dynamics of something with a larger user base being more trustworthy don't change because we're on the internet. It's a simple network (no pun intended) effect.
If the program "can download custom instructions after installation" then a large user base still means that users will be bound to find it sooner or later (and make it public/fix it) and it makes it less likely to be you (because you're 1 from 200,000 downloading it, not 1 of 20 or 200).
It's like being emailed a .bat file in 1999. Would you blindly run it on your Windows box?! I hope not!
It's funny how so many obvious things are all but obvious when you think a little bit more about it. Interesting read on the subject: https://sandstorm.io/news/2015-09-24-is-curl-bash-insecure-p...
(I don't want to enter the `curl | bash` good or bad rabbit hole; just that the topic cannot be just dismissed as "obvious")