Re:Shoestring Budget? Starting to feel growth issues on you...
by SwellJoe on Tuesday September 09, 2008 @02:31PM
(#299684)
It's also true of Perl. But it wouldn't have solved the problem in this case...a blocking IO single-process vs. non-blocking multi-process configuration is obviously not going to fare well, and it couldn't be made (significantly) faster by rewriting some bits and pieces in C.
There are also pretty big costs associated with shell scripted processing, like spawning a new shell for every command (while Perl, Python, etc. do not, unless you're explicitly forking). While there was a thread about programmers greps a few days ago, and out of curiosity I compared find+grep to grin and ack, and found that find piped to grep required a quarter the time of ack (written in Perl) and more than an order of magnitude less time than grin (written in Python) to perform a simple search, I also know that for more complex cases the cost of shell-based processing grows as complexity grows.
For an interesting comparison, look at the GNU autotools vs. cons (in Perl) or SCons (in Python). Both cons are dramatically faster than the autotools. Partly because they aren't as complex and don't target quite as many build systems, but also partly because they do most everything from a single running process.
There are also pretty big costs associated with shell scripted processing, like spawning a new shell for every command (while Perl, Python, etc. do not, unless you're explicitly forking). While there was a thread about programmers greps a few days ago, and out of curiosity I compared find+grep to grin and ack, and found that find piped to grep required a quarter the time of ack (written in Perl) and more than an order of magnitude less time than grin (written in Python) to perform a simple search, I also know that for more complex cases the cost of shell-based processing grows as complexity grows.
For an interesting comparison, look at the GNU autotools vs. cons (in Perl) or SCons (in Python). Both cons are dramatically faster than the autotools. Partly because they aren't as complex and don't target quite as many build systems, but also partly because they do most everything from a single running process.