It's also true of Perl. But it wouldn't have solved the problem in this case...a blocking IO single-process vs. non-blocking multi-process configuration is obviously not going to fare well, and it couldn't be made (significantly) faster by rewriting some bits and pieces in C.

There are also pretty big costs associated with shell scripted processing, like spawning a new shell for every command (while Perl, Python, etc. do not, unless you're explicitly forking). While there was a thread about programmers greps a few days ago, and out of curiosity I compared find+grep to grin and ack, and found that find piped to grep required a quarter the time of ack (written in Perl) and more than an order of magnitude less time than grin (written in Python) to perform a simple search, I also know that for more complex cases the cost of shell-based processing grows as complexity grows.

For an interesting comparison, look at the GNU autotools vs. cons (in Perl) or SCons (in Python). Both cons are dramatically faster than the autotools. Partly because they aren't as complex and don't target quite as many build systems, but also partly because they do most everything from a single running process.

This also seems to be true with Lua. I'm relatively new to the language, but in many ways it feels like a minimalistic Python, intended to be embedded in a C program as a library (though it's usable on its own as well).

Most of the language is closely tied to "tables", which are very similar to Python's dictionaries. Unlike Python, however, it has real closures and tail call optimization.

Lua is semantically more similar to Scheme, according to the language designers. The resemblance is quite intentional. Ease of embedding into C is a first-priority of the language implementation. It's almost trivial to interface to C, and in many cases it's easier to write bindings to the Lua API than to massage SWIG generated interfaces.

"userdata" are first-class values in Lua. They are containers for pointers to arbitrary C structures, and by overloading their behavior it's extremely easy to interface between C and Lua.

Lua tables are extremely elegant, and are the basis of object and class systems, all data structures, namespaces, packaging, and libraries.

The Lua interpreter is quite fast by itself, and with the LuaJIT written by Mike Pall your speeds can be even greater. The rumor is that the first public release of LuaJIT 2.0 is coming this winter... that one is a tracing compiler based on the same concepts as Tamarin, SpiderMonkey, and V8.

Yes. It seems to have many of the same abilities as Scheme (and I forgot to mention it has coroutines.), though it doesn't have read / the sexp syntax or macros. It's extraordinarily portable, though, and also seems to also contain most of what I like about Python (though it lacks its remarkable libary).

I read The Implementation of Lua 5.0 (www.tecgraf.puc-rio.br/~lhf/ftp/doc/jucs05.pdf, 15ish page pdf) last week, experimented some with _Programming in Lua_ 2nd ed. since, and I've been very impressed with how tasteful the overall language design seems. It seems like a brilliant tiny language, much like Forth and its descendants. (I've read that the Lua source is good reading, too. See: http://www.reddit.com/comments/63hth/ask_reddit_which_oss_co...)

The Lua Emacs mode is nothing special, but I'll see if I can improve it. :)

Python has no advantage in this regard over any other language supported by SWIG. And perl almost certainly has more available library bindings than python.

Someone hasn't heard of ctypes: http://docs.python.org/lib/module-ctypes.html

It lets you use arbitrary c libraries in Python. It automatically does the wrapping for you.

I hadn't heard of ctypes, but the same functionality is available in perl. It seems pretty silly and crash prone to me to dynamically load object code and then wrap it with a load of error checking rather than write proper bindings.

ctypes, cython, Boost::python and pyrex are real advantages in this regard; SWIG is extremely painful in comparison.

Both Perl and Python have far nicer ways to link to C than SWIG. Having worked with both, I think they're about equal these days on the binding to C front (though, as always, in Perl there's far more than one way to do it, and there's only a couple of obvious ways to do it in Python).

Do you know of any perl & C++ solutions other than perlxs and swig?

I've never used C++ from Perl, only C, but there is a version of Inline for C++ here: http://search.cpan.org/~neilw/Inline-CPP-0.25/

The nice thing about Inline is that the bindings are obviously named (function foo() in C becomes sub foo in Perl), and it's all automagic. Of course, if your C/C++ code is big or an existing library you want to bind to rather than just a few bits and pieces, perlxs (and its related utilities) might very well still be the way to go. There is XSpp, which supposedly makes C++ and XS play nicer together. But perlxs is dramatically better than SWIG in most circumstances I can think of (I don't mean to imply that SWIG is not a wonderful tool--it's a fantastic tool, but both Perl and Python have language-specific solutions to most of the same problems that are superior).

But you can't beat Inline::C (and I guess Inline::CPP) for simplicity. If you know both Perl and C/C++ their usage is absolutely obvious. perlxs requires a little more reading, and I have to consult the docs whenever I cross paths with it. Python has Weave (part of SciPy), if you like the inline model, though I'm not sure it's being maintained heavily these days.

If large data sets are your problem (some of the things you might use the numeric data structure tools in Boost for), PDL might be a solution, since it has a lot of tools for manipulating large data sets without having to revert to writing your own C. Likewise NumPy in Python. And, of course, there are already Perl bindings for Boost in CPAN, though the vast majority of Boost seems to be just making C++ more like Perl, rather than adding anything that Perl coders would find interesting.

But python is a pleasure to code. Also, I prefer Boost, and know the python bindings are solid.