News for hackers. Stuff that matters.

A comment on Introduction to sed. Parent comment

Re:Introduction to sed by _delirium on Friday October 07, 2011 @02:13PM (#3085167)
On the speed question, I actually find Perl considerably faster than sed in a lot of use-cases, if there's enough processing to dominate the slightly higher startup costs of Perl.

For example, at one point I had reason to take a gigantic single-line textfile, and break it into lines based on a specific 3-letter pattern that didn't occur anywhere else:

   s/ABC/A\nC/g
In whatever sed comes with Debian, this took about 10 minutes, CPU-bound, for a 2-gigabyte file. With Perl: 1.5 minutes, IO-bound. Not too sure why. Maybe sed runs everything through the regex engine, while Perl special-cases constant strings? Perhaps Perl has better buffer management for processing gigabytes of text? I haven't done any real testing.
[ Parent | on HN ]
Re:Introduction to sed by dbbo on Friday October 07, 2011 @02:26PM (#3085227)
The copyright section of sed(1) suggests that the Debian package sed (I'm running wheezy) is Gnu sed 4.2.1[1]. Plan9 sed is also available in the 9base package. I'm running perl 5.12.4 (also from Debian testing). I don't see any significant difference in speed with vanilla 5.14.1 and the commands I mentioned earlier.

[1] http://www.gnu.org/software/sed/

[ Parent | on HN ]
Re:Introduction to sed by dramaticus3 on Saturday October 08, 2011 @07:25AM (#3087681)
Turn on locale UTF-8 for GNU software before benchmarking it to compare with Plan9 software.

GNU stuff gets a speed improvement from assuming single byte characters.

[ Parent | on HN ]