And I'll believe Maglev's outlandish performance claims if and when I ever see them realised. If they are anything like those claimed at RailsConf 2008 (50x or thereabouts), I'll eat my hat and any other hats anyone cares to send me.
I say this as someone who would probably seriously consider murder if it would make my Rails apps run 10x faster. I want it to be true. I just don't think it is, unfortunately.
And one could probably take just about any Lisp and give it the power of the other 10% (like Erlang) with moderate effort.
I keep hearing things like this, but no-one ever does it.
And actually, I know that some Smalltalk routines have run faster than the same routine implemented in C. By 3% actually. This was over 5 years ago, and it was a block cipher -- yes, low-level bit slinging. The DLL was the RSA reference implementation. (1)
Also, you should be careful about reasoning about benchmarks as if they applied the same across the board. Anyone who knows a little about them knows that they are very contextual.
BTW, in the Smalltalk community, spelling the language name with a capital T is considered a mark of the PHB.
But if you want a Smalltalk library that's much faster than Ruby 1.8, there are a variety of benchmarks around. Just take some of that code, and declare it to be a library. QED.
(1) - This is the magic of generational GC. if you know what you're doing, this is like getting a Buffer Cache you didn't even have to implement. Also, the implementor got the help of the VM engineer, who provided primitives for handling 32 and 64 bit arrays.
I'll try to restate. You said this:
"There are Lisp and Smalltalk VMs out there that are something like 70X as fast as Ruby 1.8!"
That is what I was trying to argue with. I have never seen any proof, beyond a few cherry-picked examples of Ruby at its worst and Smalltalk at its (presumable) best, where anything like a 70x speed difference is observable. On average, ST (is that ok?) seems to be a mere 5x as fast (or so), coming in at somewhere between 5 and 10 times slower than C.
Ruby 1.8 is in the average case around 30x slower than C. So if Smalltalk can be 70x faster than that, then it is generally twice as fast as C, on average, in everyday use. This is not the case, as far as I am aware. I see this "Smalltalk is SO fast" meme again and again but it just doesn't seem to be grounded in reality.
Sorry for my idiotic error earlier, and for sounding like a PHB ..
The real gain for Ruby, once there are reliable VMs that have good JIT capabilities is the following:
A dynamic web page can then be compiled into a JIT-ed method, with static HTML as data in arrays interspersed function/procedure calls. At this point, you can increase the JIT memory cache so that it can contain your entire web site, and at that point your web pages are machine language routines!
VisualWorks has a framework that does this. It's called Smalltalk Server Pages.
ST is kosher.
If it was, it would be faster - much faster - than C, and if that was the case, games, graphics drivers, hell, operating systems - would be using Smalltalk. They are not. QED.
Smalltalk is great and its VMs are highly advanced. I accept that. I still use Ruby and eat the performance cost because I prefer its syntax and "world". But man, let's keep the performance comparisons grounded in reality. 70x faster is ludicrous.
My point is that VMs that have been around for decades capable of supporting languages of equivalent power, and that these are some silly multiplier faster than Ruby 1.8. This is going to be something like halfway to two orders of magnitude for some silly benchmarks, and much less in real life. Really, there's not much of a problem in practice, especially with the embarrassing wealth we have lying around in terms of processing power.
The real problem -- the thing I am making a point about -- is people going around saying that, "Oh well, you have to give something up to get the power..."
No. You. Do. Not!
Ruby is not that advanced. Yes, I would say that it's a lovely language. It has everything I want, but it's not doing anything that hasn't been done before. It's unique in that it combines lots of elements in a way that's really dandy, but there's nothing magic about it. There's certainly nothing so esoteric, that it couldn't be made to run fast. (There are a few things that were just bad design decisions, like making every Object behave like a little hashtable, which could've been left out and made it much easier to run faster without losing power and expressiveness.)
Oh, and people did implement OSes in Smalltalk and Lisp way back when. Also, ever hear about something called Singularity? That's a recent OS built on top of a VM. In fact, I think that more people should be doing just that -- building OSes that are "turtles all the way down" and built to not only support a particular VM, but to have every single atom of it available to tweaking by a high-level language. (Lua would be great for this.) But this is a digression.
Again, going back to my original point -- the real ludicrous thing is that the programmer community as a whole is still in the thrall of expectations about speed vs. programming language power that date from the 70s. And anyone who thinks you have to give up as much speed as Ruby has to get that power is very much in the thrall of such an idea.
I wasn't claiming you can't tack feature X onto lisp, of course you can, you can implement anything in anything. The point I wanted to make was that "power" is not just some object feature bullet points, it's about ease of use and syntax and libraries as well.
"If you add Traits (library for Mixins) to Smalltalk, it is equivalent to Ruby in power. (1)"
Only for your definition of "power".
Yes, everyone loves Ruby's super-flexible objects and mixins etc but that is only the start of the story. Its syntax is terse, intuitive and good-looking. The rubygems packaging system is effective and ubiquitous. Irb is excellent and "live coding" techniques are invaluable. To me, these are very much part of the real "power" and the main reason I prefer it over, say, Python. You might be able to emulate the object behaviour but it's still "incomprehensible mess of brackets" Lisp.
You might be able to claim that Lisp is 90% "homomorphically equivalent", a phrase I don't think I've heard since Mathematics 2, but who gives a shit? Go ahead and add the last 10% of whatever it is that Ruby has and Lisp doesn't. Now you have just as much power as Ruby? So go ahead and type "require activerecord" (don't forget to compile!) and just try to tap into that power. Oops.
"Language power" is not just some reductive theoretical idea of what features a language has, or could implement. It's the whole package, and how you use it. Sure you can add Ruby's features to C. But a better way is to implement Ruby in it.
Lisp might well be the infinitely flexible meta-language which can be used to describe all others, but that isn't actually useful in itself, as used today. Lisp may be an incredibly powerful tool for something, but it has little advantage for conventional end-user code, not any more anyway. It's pretty exciting to think where it might end up, though.
"And one could probably take just about any Lisp and give it the power of the other 10% (like Erlang) with moderate effort."
Again, only for your definition of "power", which is not what the average programmer who is just interested in getting stuff done is thinking about.
Well, for those who actually have a little intellectual curiosity, the implications are:
1) Not only can you do anything in langauges X and Y
that you do in Ruby, you can express it about as
concisely
2) Any libraries in Ruby could be just as beautifully
expressed in languages X and Y
3) There's no reason something that closely resembles
Ruby but runs *a lot faster* shouldn't exist
4) Speed is just one factor among many. There are lots
of other things Ruby could learn from.
(For the others, Ruby is awesome, it is the ultimate, there shall be nothing better, yada yada. Now you can rest easy.)
There has been a bajillion hours of awesome work that's gone into Ruby and Rails. There are ways that Ruby and Rails can be changed such that the next bajillion can be several times more awesome. All I'm saying is -- look around and be curious. There's a lot out there to learn!
The one of the biggest misconceptions we have to face in the Computer programming and related professions, is that programmers are on the forefront of technology. Actually, culture lags technology. And in that, I specifically include the culture of computer programmers.
The foundations of OO languages like Ruby and Python were set forth in the 70s and 80s. WYSYWIG word processing, ethernet LANs, desktop publishing existed then in pretty much their current forms at Xerox PARC, only slower. The concepts embodied in Dot-NET and Microsoft's Singularity were bandied about in the 60s, 70s, and 80s.
The sad thing about our profession is, is that to learn what's coming next, and what's possible you still have to do some digging into history. Things have improved, but because programming is also a cultural artifact, it lags technology like any other cultural artifact.
I keep posting this. I used to wonder why it seemed like tilting at windmills. After writing this post, I now realize why.
Alan Kay: "The Computer Revolution Hasn't Happened Yet."
http://video.google.com/videoplay?docid=-2950949730059754521
(1) - In fact, it is homomorphically equivalent. This is precisely why something like Maglev can work. Another interesting thing -- many Lisps are also homomorphically equivalent to at least 90% of the languages out there. And one could probably take just about any Lisp and give it the power of the other 10% (like Erlang) with moderate effort.