News for hackers. Stuff that matters.
Re:Making Debian Responsible For Its Actions by heresy on Tuesday September 28, 2010 @06:14AM (#1735166)
Why should the author report a bug when it's the incompetence of the Debian maintainer?

The 'gem' command not working after installing the 'rubygems' package does seem like a bit of a bug, to put it mildly, you'd think a cursory test by the maintainer would catch that.

Re:Making Debian Responsible For Its Actions by davidw on Tuesday September 28, 2010 @06:19AM (#1735178)
> Why should the author report a bug when it's the incompetence of the Debian maintainer?

Because he cares about the bug being fixed, rather than just whining? The point is to focus on fixing the system, not on pointing fingers or telling eachother to fuck off.

The maintainer most likely already had the ruby-ssl package, and so the 'cursory test' worked for him.

Look, I don't think the rubygems package on Debian is a fantastic bit of work, but ultimately:

It's free software. If you don't like it, either help fix it or don't use it. Point it out if you like, but don't be such a dick about it.

Re:Making Debian Responsible For Its Actions by heresy on Tuesday September 28, 2010 @06:42AM (#1735223)
It's clear we disagree.

The author put something out there under a free license, Debian benefits by including it in the distribution.

There is no onus on the author whatsoever.

People bitch because it's always Debian from which these problems originate.

Mac OS X - Fine. FreeBSD - Fine. RedHat/CentOS - Fine. SuSE - Fine. Debian - Commands missing, functionality disabled, misconfigured.

Re:Making Debian Responsible For Its Actions by davidw on Tuesday September 28, 2010 @06:45AM (#1735228)
The author put mongrel out there under a free license, as well as some unnamed gem, and various other things. The gem is not included in Debian. As to the "benefits" of Debian including anything, they certainly aren't pecuniary, but accrue mostly to the end users (or not, in this case). So, if you help Debian fix something, the beneficiaries are ultimately all the users of Debian, not really 'Debian' itself.

> Mac OS X - Fine. FreeBSD - Fine. RedHat/CentOS - Fine. SuSE - Fine.

Any sufficiently complex system, including every one of the above, has problems. They may differ from Debian's in type, nature and even quantity, but they exist. Including missing commands, disabled functionality, and misconfiguration.

Re:Making Debian Responsible For Its Actions by papaf on Tuesday September 28, 2010 @06:56AM (#1735250)
Why should the author report a bug when it's the incompetence of the Debian maintainer?

    1. Its the polite thing to do.
    2. The bug can be tracked and commented on.
    3. The bug is more likely to be fixed.
    4. Thats how most open source projects work.
I've not used Homebrew on the Mac, but it does look like an promising alternative approach. If you don't like something, you can fork the distro, fix the bug and put in a pull request yourself.
Re:Making Debian Responsible For Its Actions by regularfry on Tuesday September 28, 2010 @06:29AM (#1735192)
As far as I'm concerned, having rubygems available at all in Debian is a bug. Gems should be repackaged as .debs.
Re:Making Debian Responsible For Its Actions by prodigal_erik on Tuesday September 28, 2010 @06:41AM (#1735219)
This. The point of Gems and Maven and the CPAN client and all this other crap is to smuggle code into a production box while carelessly failing to get it into the platform's database of what's installed where and what depends on it. They replace one tool that knows everything that's going on, with many tools each of which can't.

Somehow we got stuck with a critical mass of small-scale developers who never faced the nightmare of somehow keeping the right tarballs on hundreds of machines in different roles without good tools, so now we all get to relive it.

Re:Making Debian Responsible For Its Actions by cemerick on Tuesday September 28, 2010 @07:56AM (#1735371)
That's hilarious. Users of Maven and CPAN represent a "critical mass of small-scale developers"? Holy shit.

A similar idea was voiced a few days ago, specifically regarding Java libraries (http://fnords.wordpress.com/2010/09/24/the-real-problem-with...).

My response remains the same: Debian (or what-have-you) is but one distro among many, and those of us that treat such things as important but fundamentally commodity deployment platforms have no wish to spend our lives babysitting whatever package manager happens to be on the operating system we're using that month/year/whatever. Further, plenty of people have plenty of good reasons for deploying on Windows and OS X, so using tools that work well there make sense.

The priorities of the various linux distros aren't everyone else's priorities – which somehow seems to come as a galloping shock to some.

Re:Making Debian Responsible For Its Actions by billswift on Tuesday September 28, 2010 @04:19PM (#1737116)
That is why I use Slackware. Installing new software by hand from tarballs is a bit tedious compared to package managers, but it is transparent, and when I am done I can be pretty sure it is going to work. I have tried other distros, even Debian once (for some reason the Debian wouldn't let me sign in as root, I never did figure out why), but I keep coming back to Slackware - for its transparency.
Re:Making Debian Responsible For Its Actions by koenigdavidmj on Wednesday October 20, 2010 @11:26AM (#1811563)
And Slackware's packaging is light enough that it is fairly simple to write wrappers around gems/CPAN modules/Python eggs. The latter two have prewritten SlackBuilds that you can use, assuming that the setup.py is not doing anything too shady.
Re:Making Debian Responsible For Its Actions by phaylon on Tuesday September 28, 2010 @11:18AM (#1735922)
With CPAN (and local-lib and/or perlbrew) I can have per-user libraries, per-application libraries, and it doesn't interfere at all with the dependencies of the system. If a library that both my application and the system use, and they fix a bug that includes an incompatibility, I can upgrade my applications without fiddling with the system, I don't need to take care to break APT in the process (fun times). I'm not even talking about possibilities like copying someone else's local-lib to my own environment to help them debugging, while still having my own setup stay untouched.

So, I guess I disagree. These tools provide a lot more than the usual package management.

To me it seems backwards to try to manage libraries globally when their use-cases are often already localized.

Re:Making Debian Responsible For Its Actions by davidw on Tuesday September 28, 2010 @06:42AM (#1735224)
One hack might be to have rubygems do that automatically and transparently on Debian.
Re:Making Debian Responsible For Its Actions by rbanffy on Tuesday September 28, 2010 @07:02AM (#1735261)
Since gems are packages themselves, why can't some clever rubista write the missing gem2deb and gem3rpm translators?

Why can't Zed do it and fix the problem instead of ranting about it?

I know I won't. It's not my problem. This is a WONTFIX bug for me.

Re:Making Debian Responsible For Its Actions by regularfry on Tuesday September 28, 2010 @07:35AM (#1735319)
This has been done, by several people. The problem is that the resulting packages don't fit with Debian policy, and in a few cases result in file collisions. If we want to mend the situation, we need both sides to agree that the solution is suitable.

The Right Way to do it, in my mind, is to take a snapshot of the available gems, and convert only those versions of the gems to debs such that rubygems and the Gem constant aren't involved at all. This means that "gem 'foo'" gets thrown away (or replaced with a check that the correct distro package is installed), using the Gem constant is a blocking bug, and "require 'rubygems'" is at least a no-op. Files from lib/ should be installed to /usr/lib/ruby (or at a pinch site-ruby), so that "require 'foo'" just works. Anything that assumes File.expand_path(__FILE__,"..") is under that library's control is again a blocking bug (I'm looking at you, haml).

I know there are people working on this, so it's not entirely theoretical, but besides, these are all good ideas anyway.

EDIT: Oh yeah, dependencies. That too.

Re:Making Debian Responsible For Its Actions by FooBarWidget on Tuesday September 28, 2010 @02:37PM (#1736726)
> The Right Way to do it, in my mind, is to take a snapshot of the available gems, and convert only those versions of the gems to debs such that rubygems and the Gem constant aren't involved at all.

If you think that's going to work, you don't know enough about the Ruby library ecosystem. You can't take a snapshot of the available versions and think that'll satisfy users. Users need to be able to install different versions of the same package in parallel, tons of things rely on this. Take a look at the rationale behind Bundler: http://gembundler.com/rationale.html Sadly dpkg doesn't support multiple package versions.

Re:Making Debian Responsible For Its Actions by regularfry on Wednesday September 29, 2010 @01:46AM (#1738431)
Oh, I know about the Ruby ecosystem. Having to rely on multiple versions of the same library is a bug, not a feature. People do it because the tool lets them, not vice versa.
Re:Making Debian Responsible For Its Actions by saurik on Friday October 01, 2010 @08:36AM (#1746823)
tl;dr dpkg models this just fine, thank you very much ;P

I don't feel it is accurate to say "dpkg doesn't support multiple package versions"; let's say it let you install multiple versions of the same package at once: now you are just going to have file-on-file conflicts. From dpkg's perspective these different versions of the same files, which live in different places on the filesystem and can coexist on the same system, are just different packages.

The real question I feel the Ruby community should be asking, and thankfully the very question that I have personally heard from Yehuda, is "why doesn't C feel the need to do this?". The answer is "well, they do, but".

For the "well, they do" part, we can easily call into evidence that on my system I have both libreadline 5.x and libreadline 6.x: there is a (hilarious) fundamental break between these two versions of the library, leading to them deciding to increase the major version number, which then causes different applications to need to continue linking to one or the other.

Therefore, Debian models this: these two major versions of readline are each a separate package, libreadline5 vs libreadline6. You can install them both at the same time: they do not conflict.

Now for the "but", where the question becomes "what if I want to install 6.0 and 6.1" and the answer is "you don't want to do that, no one wants to do that, that is not something you should want to do", and this, I feel, is where the conversation starts breaking down with the Ruby community.

The reason no one wants to install 6.0 and 6.1 is that there is no reason to: 6.1 is newer and better than 6.0. Meanwhile, software that currently works with 6.0 /will/ work with 6.1. If software that worked with 6.0 didn't work with 6.1 it would not be called 6.1, it would be called 7.0.

Now, exactly whether the major version or the second-most major version (libxml2 uses this) or whatever other scheme someone comes up with is used to determine this isn't important. The way it works for C is that the version you are using gets fixed at compile time, which I think people should actually start thinking as analogous to creating their Gemfile.

Here's how that works (for Linux gcc with typical paths): the developer of a project passes -lreadline to the compiler. This looks for /usr/lib/libreadline.so, which in turn is a symlink to a file which is named with the "compatibility version" (in this case, 5 or 6), which in turn is a symlink to the specific version of the library in question (although there might be some more levels if someone is being silly).

/usr/lib/libreadline.a /usr/lib/libreadline.so -> /lib/libreadline.so.6

/lib/libreadline.so.5 -> libreadline.so.5.2 /lib/libreadline.so.5.2 /lib/libreadline.so.6 -> libreadline.so.6.1 /lib/libreadline.so.6.1

Then, every binary has in it an "installed name" or "soname" which is the path to the library that will be encoded in any compiled binaries that have linked against it. This name is based on the compatibility version, and not the specific version, as there is an assumption that versions of the library that have the same compatibility version are, by definition, compatible.

So, this means that even though this symlink scheme theoretically supports having 6.0 and 6.1 installed at the same time, it would have no effect: all software wanting 6.x will be linked against libreadline.so.6, which will in turn only be able to point to either 6.0 or 6.1.

This assumption is a good thing: it means that we can upgrade libraries. People should not be wondering whether upgrading to 6.1 will break their program: if it does then the person who released 6.1 is not doing their job right, and there is a bug in that library that needs to be considered critical.

Luckily, system integrators such as Debian go to great lengths to test and verify that these libraries really are as compatible as they claim. This sometimes gets them into really hot water, however, as if upstream goes nuts and starts breaking their ABI, even slightly, Debian either needs to fix that bug or not take the update: it is not acceptable to Debian to allow an incompatible release of readline into the ecosystem, as it will have unpredictable consequences on the software that is using it.

That assumption is really important: there is no "we came really close, but some random things have changed that you now need to go fix". The idea in this world is that there is no good reason for that: there could be an arbitrarily large number of people who are relying on that feature, so it is irresponsible to unilaterally decide that you can just change it, even if it makes more sense.

This belief that "incompatibilities are serious bugs" leads to a number of common C idioms. To draw a few examples: * rather than change old APIs, add new ones; * rather than changing structures every now and then, use version numbers to tell different users apart; * use C for public external interfaces over C++ (which tends to have ABI fragility problems on most platforms) whenever possible.

Given this belief, if libreadline6 comes out, you also would never even consider using it in place of libreadline5 without going back to the source code to try to verify if it works (at least compiling it, but there may also be semantic issues at work in the API). With this belief in mind the idea of using ~> (which Yehuda has to spend a lot of time convincing people of) becomes "obvious": if a major version number (which unfortunately is not what ~> correctly models, but at least it is better than nothing to get the idea out there) changes, your package should not assume that will work.

You also find yourself being really happy that there is someone in the ecosystem--Debian in this case--who is doing all of that really hard, incredibly grueling, and (apparently) often thankless job of standing at the gates making certain software doesn't just willy-nilly enter the ecosystem until it has been reasonably regression tested. The developer's goal is, sadly, not always aligned with the grand unified vision, and that's what the users of these systems are buying in to (and I'll even go so far as to say: and that is where most of the value is, not the individual software projects).

Ok, </rant>. ;P (If anyone is curious: the reason libreadline5 and libreadline6 are incompatible with each other is actually that 5.x is GPLv2 and 6.x is GPLv3; afaik, and I'll admit I might be wrong, not only is the API between these two major versions identical, but so is their ABI: only in the world of politics and legalities was there an interface break. It still makes a simple example that a lot of people have run in to, though. ;P)

Re:Making Debian Responsible For Its Actions by gaius on Tuesday September 28, 2010 @04:35PM (#1737195)
Someone needs to teach the ruby guys what "regression testing" is then.

Unless you are - for some reason - relying on a bug, there's no reason not to have the current stable release of any library.

Re:Making Debian Responsible For Its Actions by rbanffy on Tuesday September 28, 2010 @10:29AM (#1735779)
> This has been done, by several people. The problem is that the resulting packages don't fit with Debian policy, and in a few cases result in file collision

Then, by definition, it has not been done - it has been attempted and the problems proved hard.

I think the best way to go about it is leave the Debian Ruby subsystem alone and either go the virtualenv route or install it from the source tarball and place it in /usr/local or /opt

Re:Making Debian Responsible For Its Actions by regularfry on Wednesday September 29, 2010 @01:49AM (#1738433)
That's fine for development. It sucks for end users and sysadmins.
Re:Making Debian Responsible For Its Actions by thwarted on Tuesday September 28, 2010 @09:54AM (#1735670)
Here's a real brute force one I wrote a while ago that seems to handle the 90% case. It was a bear to figure out. Please fork and improve.

http://github.com/thwarted/gem2deb

Re:Making Debian Responsible For Its Actions by duskwuff on Tuesday September 28, 2010 @11:24AM (#1735939)
Gems have the bizarre feature of allowing you to install multiple versions simultaneously. This is really hard to represent in a deb. :)
Re:Making Debian Responsible For Its Actions by cmsj on Wednesday September 29, 2010 @03:34AM (#1738614)
Why do people keep representing this as bizarre when C libraries parallel install just fine and you probably all have multiple versions of the same library installed right now? Accept it's a legitimate situation please. What it needs is radical change in languages like ruby/python/etc to resolve dependencies explicitly at runtime. Currently they hide the problem with import path priorities.
Re:Making Debian Responsible For Its Actions by duskwuff on Sunday October 17, 2010 @11:51PM (#1802309)
(late, but replying anyway)

Multiple versions of a library are certainly possible with C, but are generally avoided on Debian where possible. Where that isn't possible -- for instance, for libraries which completely changed their API like gtk1.4 -> gtk2, they simply create completely separate packages for the two versions.

With gems, this isn't quite so easy, as there's no way to distinguish between a minor bugfix and a huge API-breaking change without either guessing based on the version number or having a human read the changelog. Even then, there's no guarantee that dependent packages will depend on distinct sets of versions.

Re:Making Debian Responsible For Its Actions by regularfry on Friday October 01, 2010 @03:26AM (#1746331)
Rubygems adds the gem() method, which allows you to specify at runtime which version of a dependency you want, and does not rely on path munging.
Re:Making Debian Responsible For Its Actions by rbanffy on Tuesday September 28, 2010 @01:41PM (#1736491)
How do you specify which version you want in your program?
Re:Making Debian Responsible For Its Actions by blasdel on Tuesday September 28, 2010 @06:33PM (#1737621)
The naive way will just load the highest-numbered version you have installed:

  require 'rack'
When you have rubygems loaded, this is equivalent to:

  gem 'rack', '>=0'
You can specify exact and relative dependencies easily. There are a fuckton of frameworks to handle this for you, many of them oriented around copying the depended-upon gems into your app when it's deployed, as is common in the Java world.
Re:Making Debian Responsible For Its Actions by telemachos on Tuesday September 28, 2010 @02:10PM (#1736634)
Bundler[1] is one way.

[1] http://gembundler.com/rationale.html

Re:Making Debian Responsible For Its Actions by JoachimSchipper on Tuesday September 28, 2010 @07:35AM (#1735320)
Because the gem format probably doesn't have a proper way to say "you must install OpenSSL such-and-such"? At least CPAN and Python's eggs don't have one...
Re:Making Debian Responsible For Its Actions by ekidd on Tuesday September 28, 2010 @09:32AM (#1735610)
OpenSSL is an interesting case, because it's part of the standard Ruby library, and it should be available in every single copy of Ruby. But Debian faces export restrictions on OpenSSL, so their copy of Ruby is missing a number of important standard libraries by default.

Rubygems can't really clean this problem up, because Rubygems assumes it's running on an actual copy of Ruby, and not a half-broken Debian Ruby.

To get a working Rubygems installation on Debian, I usually 'apt-get install ruby-full', which installs a non-broken Ruby interpreter, and then install Rubygems itself from a tarball so that apt-get stays _far_ away from my Ruby setup.

I don't blame Zed for being angry about this. Debian's Ruby has often been pretty broken even by vendor standards.

Re:Making Debian Responsible For Its Actions by rbanffy on Tuesday September 28, 2010 @07:06AM (#1735266)
> Why should the author report a bug when it's the incompetence of the Debian maintainer?

I think you really should go to debian.org and demand they return your money.

Re:Making Debian Responsible For Its Actions by heresy on Tuesday September 28, 2010 @07:43AM (#1735339)
I'd rather they return the youth during which I was a Debian maintainer.