News for hackers. Stuff that matters.
Re:I'm Sick of This by jrockway on Wednesday April 08, 2009 @11:21AM (#552806)
Interesting. We have sort of solved this problem in Perl. When you want to add a method to a class, you do it by applying a role. If two roles conflict, like by adding a method with the same name, you have to resolve the conflict, or the class won't compile. (The class can rename methods, but you can also call the method with a name that will always work; $instance->Role::Name::method.)

We can also apply roles at runtime. This means you can add a method to a single instance of a class. So Rails could add its version of sum to its own instances of Array, and Classifier could do the same. Then there would be no conflict, except when Classifier tries to add sum to the Rails array... but at least you get a fatal error message instead of unexpectedly wrong behavior.

We can do better, though. The way around this problem is to implement (and CPAN) a role that adds a sum method to arrays, and to have both Rails and Classifier use that role. Then they can both share arrays, and when one tries to apply the sum role, the already-applied sum role will be used instead.

They both get their sugar, the sugar is isolated to the smallest area possible (a single instance), and they can both confidently share those arrays-that-can-sum.

And oh yeah, with Moose::Autobox, you can apply roles to Perl's native types (arrays, hashes, etc.), and call methods on them. So this is not "pie in the sky", it's an already-solved problem for us :)

Edit: ContextL also provides an interesting solution, "layers". Each module can define their own layer, add their own sum method to it, and activate it inside themselves:

    (defclass array ...)

    (deflayer rails-sum ...)
    (define-layered-method sum :layer rails-sum ((self array)) ...)

    (deflayer classifier-sum ...)
    (define-layered-method sum :layer classifier-sum ((self array)) ...)


    (in-package :rails)
    (defun count-users (users-array)
        (with-active-layers (rails-sum)
            (sum users-array))) ; uses rails' sum method

    (in-package :classifier)
    (defun count-results (results-array)
        (with-active-layers (classifier-sum)
            (sum results-array))) ; users classifier's sum method

    (in-package :my-app)
    (defun broken-code (array)
        (with-active-layers (rails-sum classifier-sum)
             ;; ERROR, methods conflict
         ))
This is a pretty good approach, but not as good as agreeing on semantics in advance.
Re:I'm Sick of This by raganwald on Wednesday April 08, 2009 @11:25AM (#552822)
We can also apply roles at runtime. This means you can add a method to a single instance of a class. So Rails could add its version of sum to its own instances of Array, and Classifier could do the same.

This is possible with Ruby, but culturally rare. Quite possibly because (blamestorming mode=ON) Rails simply patches whatever it wants, whenever it wants and many people follow its lead. I personally avoid the practice because it is difficult to contain in Ruby. For example, if I am handed an Array it is easy to extend that instance with its own #sum method, and when I'm done with the array I can remove my own extension.

However, what happens if, while I am still holding this extended array, I pass the array to some other method that wants to extend it with a #sum method? Same problem. Working on an instance method does reduce the scope of each modification and thus make collisions statistically less likely, so I am all for it. But at the same time, when I step back and look at the big picture, I yearn for something that solves the problem in a more fundamental way.

Re:I'm Sick of This by jrockway on Wednesday April 08, 2009 @11:32AM (#552836)
It is interesting... Perl has such a reputation for being unmaintainable that Perl programmers today are very careful to spend extra effort making their code as maintainable as possible.

I think a lot of practicing Ruby (and Python) programmers are under the impression that Ruby magically solved Perl's maintainability shortcomings, and so they don't invest any effort into writing maintainable code. The result is ... unmaintainable code :) It turns out that programmers write the unmaintainable code, not programming languages.

Re:I'm Sick of This by raganwald on Wednesday April 08, 2009 @11:38AM (#552847)
I feel that using patterns or coding conventions to make Ruby metaprogramming maintainable is isomorphic to greenspunning higher-level programming practices with if statements and gotos. It is assuredly what a good programmer does to get the job done with the tool that is provided. But a toolsmith thinks about ways to improve the tool itself.
Re:I'm Sick of This by scott_s on Wednesday April 08, 2009 @04:10PM (#553345)
I think you've successfully avoided the problem through rigorous adherence to a design pattern.

A full solution would be to synthesize that design pattern into a language feature so its use is enforced by the language, not programmer discipline. This, I think, is what Raganwald is calling for.

Re:I'm Sick of This by nkurz on Wednesday April 08, 2009 @02:48PM (#553210)
"you can also call the method with a name that will always work; $instance->Role::Name::method."

I'm coming at this from a Perl and C background, but I'm not really understanding the fuss. It seems obvious that diddling with base classes is not scalable unless you have have some strong social conventions. But why do it at all?

For those not fluent in Perl's syntax, Jonathan's suggestion is equivalent to "$instance->Package::method()", which is in turn equivalent to "method($instance) from within a given Package. This seems foolproof. No one's followed up on it, but at a glance I don't see any downsides to this approach.

I may be illustrating my ignorance (and I'm definitely ignorant of Ruby syntax), but why is there such strong desire to have array->sum() instead of sum(array)? Or as a utility function Utility::sum(array)? And is there no equivalent in Ruby to array->Utility::sum()?

Re:I'm Sick of This by davidmathers on Wednesday April 08, 2009 @12:24PM (#552918)
"The way around this problem is to implement (and CPAN) a role that adds a sum method to arrays, and to have both Rails and Classifier use that role."

+1. I see this, ideally, as a social organization problem rather than a conflict resolution/avoidance problem. The conflict between two array#sums is a side effect of the real problem which is that each project had to create array#sum in the first place.

The granularity of gems is too large, so some other system would need to exist. That system, whatever it is, seems to me the answer.

Merb, and other projects, have "extlib" gems that go along with them. I think that's another symptom of the problem. Those need to go away and some sort of social extlib repository/system needs to take their place.

EDIT:

Facets can probably be thought of as the cathedral (non-)solution to the problem. What we need is the bazaar solution.

Re:I'm Sick of This by raganwald on Wednesday April 08, 2009 @12:29PM (#552931)
I see this, ideally, as a social organization problem rather than a conflict resolution/avoidance problem.

This requires getting all of the framework, gem, and plug-in authors everywhere to agree on standards for everything. If even one goes his own way and writes his own idiosyncratic thing that conflicts with a "standard" thing, there are going to be problems.

I just can't see this scaling, it's a Tragedy of the Commons in the making.

Re:I'm Sick of This by davidmathers on Wednesday April 08, 2009 @12:58PM (#552987)
"This requires getting all of the framework, gem, and plug-in authors everywhere to agree on standards for everything."

Let me flesh it out a little bit more.

I'm not suggesting there be a canon. I'm thinking of the way github does gems, where the gems are all namespaced with a username. There could be 20 versions of array#sum, but the one that the pioneers/influencers/cool-kids-who-write-frameworks decide to use is the one that gets used. If someone decides to go their own way, someone else will fork and fix.

Maybe conflicts could be detected at the gem level via gems exposing the names of the monkey patched methods they contain.

Honestly I think git/github fundamentally changes some things and that we as a culture haven't yet fully adapted our thinking to the new possibilities.

Of course there will always be problems, and maybe this is utopian crazy talk, but the natural "that can't possibly work" reaction might be worth questioning.

EDIT:

Allow me to go from just crazy to total nutter: a language market where the commodity is semantics and the currency is popularity. If it were given influence over language design would it be an improvement over the benevolent dictators (Larry/Matz/Guido) or an epic fail?

Re:I'm Sick of This by davidmathers on Wednesday April 08, 2009 @03:17PM (#553265)
Arg, more crazy thoughts. But, a (rhetorical) real world question:

Once pasta chef Katz is done turning the rails spaghetti into merb-style ravioli and all the rails monkey patching is self contained in an extlib style gem, how many projects will begin including that monkey patching as a dependency?

If someone wants an array#sum there's a good chance they'll just depend on rails' array#sum. With its massive influence the rails project has the, perhaps small, potential of becoming an autocratic second-level Matz with a default set of widely used ruby extensions. What Facets wants to be but isn't.

Given that scenario, is it better to let rails be the guardians of a defacto "standard" extlib or to have a more fine-grained system that allows the marketplace of popularity to decide what gets to be considered standard?

Re:I'm Sick of This by davidmathers on Wednesday April 08, 2009 @05:12PM (#553424)
Ok, my last crazy thought for the day:

With rails recipes the division of labor between programmers and "editors" has already begun. The granularity of the web framework has changed from the single monolithic option handed down by the programmers to the gem level. Now we get a default framework from the programmers and dozens or hundreds of remixed frameworks provided by people with specific configuration needs.

These people are doing the same job that the creators of linux distributions are doing. It's up to them to resolve compatibility issues and create any patches needed to make their distribution work. You find the custom framework that does exactly what you need or if it doesn't exist you create it and become an editor yourself.

Now take the component granularity down to the level of a single language extension or monkey patch. Gems might have gem recipes. The authors of the classifier gem would require array#sum and identify a default version, but an editor could change that.

Right now you can create a recipe that replaces prototype with jquery. Maybe someone thinks that rails' #try sucks so they create a recipe that replaces it with raganwald's #try. It's then the job of the editor to make sure all the components of their recipe play nice with each other.

Just like there are more mutual funds than there are stocks you get an explosion of recipes, but that's exactly what you want. Mass customization. You look through the directory and find the package/distribution that does exactly what you need. The software that gets used gets used.

The key here is that no one needs to agree on anything. Coders code and editors edit. The producer->editor->consumer relationship is natural. The producer->consumer relationship is artificial. It's a bug in the system.