News for hackers. Stuff that matters.
Re:Detecting the use of “curl – bash” server side by kenny_r on Wednesday April 20, 2016 @05:41AM (#11533218)
For me personally, the security aspect isn't even the reason I dislike curl | bash installers. There's no standard behaviour for curl|sh, and it re-introduces many problems that were solved by package managers decades ago.

Each time I'm about to run `curl https://install.newhotness.io|sudo sh` I'm left with the following questions:

* Will it work? I'm on Arch Linux. This installer was probably tested on OSX and Ubuntu and deemed "portable". I've seen curl|sudo sh installers trying to apt-get installing dependencies.

* Where does it place files? I've got a ~/.meteor folder that's 784MB. Why would you install software libraries and binaries to $HOME? I now have to tell my backup tool to ignore it when backing up my home partition, great. The FHS was established for a reason, people.

* Correlary to the previous question: How do I uninstall it? Maybe all files were installed to a hidden folder in ~/, maybe there's some stuff in /opt, or /usr/local, or who knows where else. If the installer doesn't implement uninstall functionality itself, I now have to go hunt for the stuff it placed in my filesystem.

I understand the need to be able to distribute your software without having to implement native packaging for OSX/Debian/Ubuntu/Red Hat/Fedora/Arch/...

Docker solved this problem. If you're feeling lazy when it comes to distributing your app, just ship a Docker container and be done with it.

Re:Detecting the use of “curl – bash” server side by TeMPOraL on Wednesday April 20, 2016 @06:04AM (#11533291)
> Correlary to the previous question: How do I uninstall it? Maybe all files were installed to a hidden folder in ~/, maybe there's some stuff in /opt, or /usr/local, or who knows where else.

How does one uninstall stuff on Linux anyway? My Linux boxes are always like thrash cans - I add and add new stuff until it's time to upgrade either the system or the machine, so it gets wiped and replaced. Never figured out how to delete stuff in a way that will not leave a ton of things behind...

Re:Detecting the use of “curl – bash” server side by viraptor on Wednesday April 20, 2016 @06:53AM (#11533474)
Package managers deal well with files which they installed. What the app creates is your business... and for a good reason. You don't want random package upgrade / conflict to remove your data just because there's something changing about the binary.
Re:Detecting the use of “curl – bash” server side by dsp1234 on Wednesday April 20, 2016 @06:17AM (#11533330)
The following generally works for me. It removes the installed files, including configuration files. Additionally it will remove any dependencies that were only used by packagename

sudo apt-get purge --auto-remove packagename

Re:Detecting the use of “curl – bash” server side by serf on Wednesday April 20, 2016 @06:19AM (#11533344)
>How does one uninstall stuff on Linux anyway?

with a package manager suited for your system, or with the knowledge of what files the software in question touches.

Re:Detecting the use of “curl – bash” server side by RijilV on Wednesday April 20, 2016 @09:04AM (#11534024)
It's a best effort though, you're relying on the package maintainer to have tracked all the installed files and clean up anything they might have modified, which hopefully you or something else hasn't modified after the package was installed.

The packages authored by distributions tend to be pretty good, but I've seen RPMs with nothing more than tar zxf in them and no tracking of the files installed.

The major package managers expect the package to manage the artifacts installed, this works if everyone plays nice and does their job...

Re:Detecting the use of “curl – bash” server side by alphapapa on Wednesday April 20, 2016 @10:01AM (#11534416)
You're right that this requires well-maintained packages, and this is one of the strengths of Debian: its policies that require well-maintained packages. If you install a package that's part of Debian, you can count on it being well-maintained, as opposed to one put out by a random web site or developer, which may install but may not integrate well or clean up after itself.
Re:Detecting the use of “curl – bash” server side by dsmithatx on Wednesday April 20, 2016 @11:19AM (#11535047)
One point I'd like to make is that this is the fault of the developer(s) and not Linux. I've seen this problem in OSX and Windows as well.
Re:Detecting the use of “curl – bash” server side by khedoros on Wednesday April 20, 2016 @03:01PM (#11536850)
Package formats each have their own expectations. RPM likes being pointed to a source tarball, given the steps to build the project, and being given a list of the just-built files to include in the package. Those are the artifacts that RPM will manage (and they can be marked things like "doc", "config", or given specific attributes). Deb is broadly similar, but I feel like the framework is a little less controlled than the RPM one.

The manager basically just defines a framework. Packages can abuse that framework in various ways (I haven't even mentioned install triggers, to execute code when other named packages are added or removed). There are a lot of ways to get it wrong, but when the dev builds a good package, it's almost magic how nicely it works.

Re:Detecting the use of “curl – bash” server side by gerbilly on Wednesday April 20, 2016 @08:47PM (#11538875)
I often do this to find out what files were installed where.

    $ touch now
    $ install.sh
    $ find / -newer now
That is supposing you trust install.sh of course.

You can redirect the output of find to a file for later removal. You will have to filter out false positives from the 'find' output, usually dev nodes.

Re:Detecting the use of “curl – bash” server side by mbrock on Wednesday April 20, 2016 @07:25AM (#11533588)
If you're on NixOS, you just remove the package from your configuration file and run the "rebuild" command.
Re:Detecting the use of “curl – bash” server side by alphapapa on Wednesday April 20, 2016 @09:58AM (#11534404)
I just don't understand this. I've used Debian and Ubuntu for many years, and I've never had to reinstall a single installation for any reason. I use packages whenever possible, and when installing unpackaged software, I use checkinstall to make a package from it. When I move to a new system or disk, I rsync the filesystem over and boot from the new disk. There is no accumulated cruft; or if there is, it's limited to e.g. tiny config files in /etc from older package versions, which aren't a problem or even noticeable.

The closest to cruft is hidden directories in my homedir from software I don't use anymore. That isn't a problem either: they don't take up much space, and it's easy to manually delete them if I ever feel like it. If not, they are compressed and deduplicated by backup software, and they're hidden by default, so who cares?

Re:Detecting the use of “curl – bash” server side by bpchaps on Wednesday April 20, 2016 @12:49PM (#11535742)
I wish I had your luck... A lot of fixes on forums constantly suggest to uninstall first. Which sometimes actually fixes issues. Other times it makes things worse. Either way - buying a lottery ticket soon?
Re:Detecting the use of “curl – bash” server side by alphapapa on Wednesday April 20, 2016 @10:02PM (#11539140)
> A lot of fixes on forums constantly suggest to uninstall first.

A lot of people seeking and offering help on forums are former Windows users, who are used to doing that to fix problems. They just don't what else to do, so that's what they do.

And, sure, if you break a configuration file or package dependency, wiping the disk and reinstalling will fix that--but so would fixing the file or dependency. In my experience, the nuclear option is never necessary with Linux. Worst case, you boot off a CD or USB drive, mount the partition, chroot if necessary, fix it, and reboot.

> Other times it makes things worse.

That is very strange, indeed. D:

> buying a lottery ticket soon?

Haha, nope, been having this "luck" for years now, but it only seems to affect my Linux installs. :(

Re:Detecting the use of “curl – bash” server side by TeMPOraL on Thursday April 21, 2016 @10:54AM (#11542584)
To be honest, I've solved many more problems with Linux by reinstalling (usually a different distribution) than with Windows.

And just yesterday my Linux did what I've never seen an OS do before - it losts it UI clock display. The solution was simple and typical for Linux - it involved killing a random process. But at that moment I realized that we used to laugh that rebooting stuff is the Windows way of doing things. Not anymore, apparently.

;).

Re:Detecting the use of “curl – bash” server side by alphapapa on Wednesday April 27, 2016 @11:58PM (#11586494)
> And just yesterday my Linux did what I've never seen an OS do before - it losts it UI clock display. The solution was simple and typical for Linux - it involved killing a random process.

What desktop environment was that? There are many different ones available for Linux systems, and they don't all behave that way. e.g. I've never had that kind of problem with KDE3/TDE or KDE4.

Re:Detecting the use of “curl – bash” server side by bpchaps on Thursday April 21, 2016 @02:48PM (#11544441)
My favorite is still hunting down .lck files with strace because of a bad lock file implementation. Reinstalling definitely fixes that one :). Looking at you, firefox and yum!
Re:Detecting the use of “curl – bash” server side by HappyTypist on Wednesday April 20, 2016 @06:17AM (#11533331)
Your package manager should have a remove operation.
Re:Detecting the use of “curl – bash” server side by lisivka on Wednesday April 20, 2016 @07:00AM (#11533498)
Open package manager and remove checkbox at a package, then press "Apply".
Re:Detecting the use of “curl – bash” server side by SixSigma on Wednesday April 20, 2016 @08:24AM (#11533832)
/usr/local exists for a reason
Re:Detecting the use of “curl – bash” server side by Spivak on Thursday April 21, 2016 @12:00PM (#11543132)
And /opt for self-contained packages.
Re:Detecting the use of “curl – bash” server side by mioelnir on Wednesday April 20, 2016 @05:27PM (#11537870)
> How does one uninstall stuff on Linux anyway? My Linux boxes are always like thrash cans...

That is called sedimentary data storage, ie. piling new files on top of the old ones.

Re:Detecting the use of “curl – bash” server side by crdoconnor on Wednesday April 20, 2016 @07:04AM (#11533515)
>For me personally, the security aspect isn't even the reason I dislike curl | bash installers. There's no standard behaviour for curl|sh, and it re-introduces many problems that were solved by package managers decades ago.

Every time I've seen it used, curl | bash is simply a way to deal with the fact that the users all have different package managers (often with outdated software) and quite complicated workflows to get stuff set up with them because there were problems that the package manager didn't solve.

>Docker solved this problem.

Ironic. Docker is one of those applications. And they used to ask for it to be installed with a curl pipe.

They seem to have replaced it with what is effectively a script executed with human hands. Not really an improvement.

Re:Detecting the use of “curl – bash” server side by kenny_r on Wednesday April 20, 2016 @07:34AM (#11533611)
I was genuinly confused with regards to your last point.

There's a section on their website[0] there are installation instructions for various distributions using repositories and package managers. In other words, the right(tm) way of doing software distribution on Linux.

But then on a different part of their website I found the curl | sh instructions you allude to[1]. Bizarre that they still support that installation method.

To make matters worse, the big orange "Get Started" call to action button actually leads to the curl|sh installation instructions. In light of that I certainly can't blame you for not knowing about the proper distribution packages.

[0] https://docs.docker.com/engine/installation/

[1] https://docs.docker.com/linux/step_one/

Re:Detecting the use of “curl – bash” server side by crdoconnor on Wednesday April 20, 2016 @09:32AM (#11534208)
That curled script actually installs the 'proper' distribution packages. It has a lot of clever stuff to figure out which environment it's running on and then does whatever commands are necessary - including, e.g., adding keys so that the external repos where docker is hosted are trusted.

Point being that even if you install "the right way" on linux using package managers it's still a multi-step complicated process that benefits from being automated in a bash script.

Re:Detecting the use of “curl – bash” server side by kenny_r on Wednesday April 20, 2016 @10:55AM (#11534835)
Thanks, that explains a lot.

It's really a shame that you have to wrap the best practices aproach in a bad practice in order to make it convenient.

> Point being that even if you install "the right way" on linux using package managers it's still a multi-step complicated process that benefits from being automated in a bash script.

Right, because distributions want different things than software authors.

If you're a Debian/Ubuntu/RHEL maintainer, you want stable, tested versions of software that you can vouch for and support for several years.

If you're the maintainer of an application, in this case Docker, you want your users to run the latest and greatest version of your software. You can't just demand that each distro just ships the latest and greatest version of your software. So if you're Docker you end up having to do the following for each distro:

  * package the software for that distro
  * sign that package
  * serve your your own repository containing that package
  * ask your users to add the repository
  * ask your users to trust the signature
And then finally your users can install the package.

That's one of the reasons I like running Arch Linux on my workstation. The desires of the software authors, distro maintainers and users align.

  Docker Inc: "Hey wouldn't it be cool if your users could install the latest
               stable version of Docker?"
  Arch Linux maintainers: "Yeah, it would. We'll keep the latest version in the
                           repo and our users can just `sudo pacman -S docker`"
  Docker Inc: "Cool."
  Arch Linux users: "Cool."
Re:Detecting the use of “curl – bash” server side by cyphar on Thursday April 21, 2016 @12:02AM (#11539603)
You should try openSUSE Tumbleweed. It actually is faster at releasing new packages than Arch in some cases, and has much more testing and stability from my experience. But it should be noted there are good reasons to want stable software especially with Docker (which is something I actually maintain for SUSE systems) -- there's been a lot of very invasive architectural changes in the last 3 or 4 releases. And all of them cause issues and aren't backwards compatible or correctly handled in terms of how to migrate with minimal downtime.
Re:Detecting the use of “curl – bash” server side by crdoconnor on Wednesday April 20, 2016 @11:32AM (#11535166)
Arch is far from stable enough to use on servers, hence why it's almost never used on servers.
Re:Detecting the use of “curl – bash” server side by kenny_r on Wednesday April 20, 2016 @01:05PM (#11535879)
I don't disagree.

You rarely see curl|sh installers for server stuff though. I think the topic at hand mostly concerns workstations.

Re:Detecting the use of “curl – bash” server side by lgessler on Wednesday April 20, 2016 @08:13AM (#11533777)
FWIW, this is Meteor's justification in /usr/local/bin/meteor:

    # This is the script that we install somewhere in your $PATH (as "meteor")
    # when you run
    #   $ curl https://install.meteor.com/ | sh
    # It's the only file that we install globally on your system; each user of
    # Meteor gets their own personal package and tools repository, called the
    # warehouse (or, for 0.9.0 and newer, the "tropohouse"), in ~/.meteor/. This
    # means that a user can share packages among multiple apps and automatically
    # update to new releases without having to have permissions to write them to
    # anywhere global.
Re:Detecting the use of “curl – bash” server side by TallGuyShort on Wednesday April 20, 2016 @09:43AM (#11534293)
I used to do Linux packaging full-time and constantly hit limitations in what Linux packaging formats support or allow. There's still a lot of progress that could be made in that area, but unfortunately any attempt to improve will likely become that 15th competing standard XKCD refers to...
Re:Detecting the use of “curl – bash” server side by pbhjpbhj on Wednesday April 20, 2016 @05:26PM (#11537867)
They could be in ~/.local/share/meteor following "XDG Base Directory Specification" (https://specifications.freedesktop.org/basedir-spec/basedir-... - seems well used on my Kubuntu but it's a case of "pick a standard" I guess).
Re:Detecting the use of “curl – bash” server side by alphapapa on Wednesday April 20, 2016 @09:53AM (#11534363)
For dealing with this problem, as well as anything that's not already packaged, the checkinstall utility is very helpful. It intercepts filesystem calls and builds deb/rpm/tgz packages that you install with your distro's package manager. It's been packaged in Debian for a long time. Highly recommended.