News for hackers. Stuff that matters.
Re:Sed – An Introduction and Tutorial by narrator on Wednesday January 07, 2015 @06:21PM (#8854001)
I just use | perl -pi -e 's/foo/bar/g' , etc for this kind of stuff. Is there anything I can't do with perl on a line that sed will do? I can see how perl is a lot more complex than sed, but I went through the whole perl learning curve back in the late 90's so it doesn't bother me that much..
Re:Sed – An Introduction and Tutorial by dogweather on Wednesday January 07, 2015 @06:42PM (#8854091)
For me, this is the whole point of perl: You can learn one tool which replaces three, each with their own syntax: Sed, Awk, and Grep.
Re:Sed – An Introduction and Tutorial by cogburnd02 on Thursday January 08, 2015 @02:33PM (#8858368)
Wait, why not just use awk? It can do pretty much everything the other two do (and can, like sed, be used to replace other things (like 'head')) and runs faster than perl at some things. (https://news.ycombinator.com/item?id=8858342)
Re:Sed – An Introduction and Tutorial by dogweather on Thursday January 08, 2015 @04:46PM (#8859226)
I learned awk in school, and like it. But I like to choose tools which increase programmer-efficiency before machine-efficiency. Perl is just one tool and one easy syntax to learn. Its regex language is now the standard as well.

Only if runtime efficiency turns out to be too slow would I re-examine choice of tools. This is like avoiding premature optimization.

(I've never found perl to be too slow in practice, btw.)

Re:Sed – An Introduction and Tutorial by scdlbx on Wednesday January 07, 2015 @10:00PM (#8854810)
It's a lot easier to delete specific lines using sed. Also you can have sed do replacements to the n'th instance of something. Doing that in Perl is a bit more complicated and a lot less succint.

$ echo "foo foo foo foo" | sed 's/foo/bar/3'

foo foo bar foo

Re:Sed – An Introduction and Tutorial by raiph on Thursday January 08, 2015 @02:41PM (#8858424)
The Rakudo Perl 6 compiler is still immature and slow, and the -i option (in-place edit) hasn't yet been implemented, but, at least for comparison's sake:

$ perl6 -pe 'next if ++$ == 2' example.txt

... prints all lines except line 2.

This is an example from Perl 6 One Liners[1].

The `$` is just just an unnamed variable that is getting incremented once per evaluation (-e is for `evaluate`) which in this case happens once per line (-p is for printing each line of input after eval'ing the code -- unless a `next` applies, in which case that line gets skipped).

And...

$ echo "foo foo foo foo" | perl6 -pe 's:3rd/foo/bar/'

... replaces the third foo with bar.

P6 regexes are far easier to read and way more powerful than P5 regexes. The `:3rd` bit is a general language feature called "Adverbs", in this case applied to the regex focused s/// built in.[2]

[1] https://github.com/sillymoose/Perl6-One-Liners

[2] http://doc.perl6.org/language/regexes#Adverbs

Re:Sed – An Introduction and Tutorial by cogburnd02 on Thursday January 08, 2015 @05:56PM (#8859596)
I've read literally nothing about perl 6 but what David Skoll wrote here: http://david.skoll.ca/blog/2010-07-29-perl-sss.html

Quote:

"I asked on a forum what the goals are for relative size and speed of Perl 6 vs. Perl 5, and a Perl 6 developer responded that a reasonable goal would be to have Perl 6 be twice as big as Perl 5 and take twice as long to start up.

"To achieve this goal, the Perl 6 developers will have to shrink the program size by a factor of 6.1 (that is, get rid of about 84% of the code.) They'll need to reduce startup memory consumption by a factor of 13.7 (that is, cut out 93.7% of their memory use) and reduce startup time by a factor of over 275.

"Oh, and this is after they add in all the missing features required to bring Perl 6 up to production-level."

Has the situation gotten better since 2010?

Re:Sed – An Introduction and Tutorial by raiph on Friday January 09, 2015 @11:47AM (#8862988)
> Has the situation gotten better since 2010?

Not really. Startup uses about the same RAM. It's about 10x faster.

The best docs I know about performance would be http://pmichaud.com/2012/pres/yapcna-perflt/slides/slide17.h... and http://jnthn.net/papers/2014-yapceu-performance.pdf#page=72

> "... all the missing features required to bring Perl 6 up to production-level."

The latest story is that the last major missing features (Unicode grapheme-by-default and native arrays) will land in the next few months and Perl 6 will be declared "officially ready for production use" by the end of 2015.

Re:Sed – An Introduction and Tutorial by emmelaich on Thursday January 08, 2015 @08:54AM (#8856335)
deleting (i.e) not printing lines Here are the examples for not printing the 4th line:

sed: sed -n '4!p'

awk: awk 'NR != 4'

perl: perl -ne '$. != 4 && print'

Not much between them really.

Re:Sed – An Introduction and Tutorial by scdlbx on Thursday January 08, 2015 @12:34PM (#8857547)
I would consider 'sed 4d' to be significantly easier than doing the same in Perl, but I don't disagree that it's not that hard to do in either.
Re:Sed – An Introduction and Tutorial by sbi on Wednesday January 07, 2015 @06:48PM (#8854112)
Probably not, but in some cases sed and AWK might be faster. I am a big fan of AWK. It is limited in some ways that make it impractical to use for really serious programs, but it is very expressive. Look at [1, 2].

[1]: http://c2.com/doc/expense/

[2]: http://www.pement.org/awk/awk1line.txt

Re:Sed – An Introduction and Tutorial by senorsmile on Wednesday January 07, 2015 @08:56PM (#8854613)
I have tested converting sed/awk lines to perl in a few base scripts that worked on a fairly large amount of data. Oddly enough, in every case, perl 5.18 performed at LEAST 1.5x faster, sometimes as much as 3.5x faster. Obviously anecdotal evidence, but recent versions of Perl seemed to have gained some good speed.
Re:Sed – An Introduction and Tutorial by _delirium on Thursday January 08, 2015 @12:15AM (#8855096)
I've had a similar experience, with Perl up to about 7x faster. I had sed in a few data-mangling pipelines because I assumed simpler=faster, but replacing it with Perl was either a wash or a speedup in every case. This with the versions of perl and sed in Debian (looks like it's GNU sed), so ymmv with other seds.

The case where I saw a 7x speedup was doing many-times-per-line, fixed-string search/replace on a file consisting of very long lines (an SQL dump where some lines had >1m characters). Perl was IO-bound (so presumably would've been even faster if I'd had better disks), while sed was CPU-bound at a pretty low fraction of the possible IO performance.

Re:Sed – An Introduction and Tutorial by Tobu on Thursday January 08, 2015 @12:01PM (#8857293)
Could have to do with charset handling.

For the really complex stuff, there's rejit[0]. I wonder if LuaJIT would work; these tools also need IO tricks to be fast.

[0] https://lwn.net/Articles/589009/

Re:Sed – An Introduction and Tutorial by ronjouch on Wednesday January 07, 2015 @09:19PM (#8854689)
Rafe Colburn from Etsy wrote about this performance oddity: http://rc3.org/2014/08/28/surprisingly-perl-outperforms-sed-...

EDIT to manage expectations: the article doesn't explain why, it just provides benchmarks and one commenter made a suggestion about character handling. More insight still welcome :)

Re:Sed – An Introduction and Tutorial by sbi on Thursday January 08, 2015 @12:20AM (#8855116)
There are many different versions of awk: gawk, (BSD) nawk, mawk, etc. I think OS X uses nawk, but mawk is reputedly faster. Gawk is definitely slower than both mawk. I'm not surprised that Perl is faster though.
Re:Sed – An Introduction and Tutorial by Tobu on Thursday January 08, 2015 @11:56AM (#8857263)
> I used the default OS X versions of these tools. The versions were Perl 5.16.2, Awk 20070501, and some version of BSD sed from 2005
Re:Sed – An Introduction and Tutorial by kerkeslager on Thursday January 08, 2015 @12:42PM (#8857609)
I'm a firm believer that there are many ways to work effectively. If Perl fills the same needs for you that sed does for some people, then by all means stick to Perl.

However, for someone who knows neither, here are some reasons you might want to choose sed over Perl:

1. sed syntax is pervasive in other tools. For example, to run a substitution from early on in the tutorial in vim, type :%s/abc/(&)/<enter> from insert mode.

2. sed is simpler than Perl. It used to be that Perl filled a unique role as a scripting language but now there are a bunch of languages in that space (most notably Python and Ruby). Python + sed for example would fulfill most of the same functions that Perl does (and there are reasons to choose Python over Perl as scripting languages, although that's a much more complicated domain to discuss).

3. sed is more performant (or so I hear). This has never been a real concern for me, but some people cite this is a concern.

4. sed usually is a bit more terse. For the length of expressions you'll typically be writing with sed, this isn't a big concern.

Disclaimer: There are probably good reasons to choose Perl over sed, too. Not being a Perl guy, I don't know those reasons. I'll leave that to someone who knows more about Perl.

When I learned sed it was because I was taking a *nix class in college, and a professor pointed me at sed and not Perl. I learned sed instead of Perl because it was put in front of me, not because of any weighing of pros and cons. That's how a lot of learning happens. Sometimes simply learning what's put in front of you leads turns out to be an obvious mistake in retrospect (I was stuck writing VBA for a little while). I don't know whether learning Perl or sed is better for what sed does, but I do know that after maybe 6ish years using sed quite frequently it hasn't turned out to be an obvious mistake.

Re:Sed – An Introduction and Tutorial by cogburnd02 on Thursday January 08, 2015 @02:28PM (#8858342)
> "3. sed is more performant (or so I hear)."

There are regexes for which awk (maybe sed too?) will perform at a reasonable speed while perl is incredibly slow. Graph: http://pdos.csail.mit.edu/~rsc/regexp-img/grep1p.png Article: http://swtch.com/~rsc/regexp/regexp1.html

Perl...could...employ much faster algorithms when presented with regular expressions that don't have backreferences.