News for hackers. Stuff that matters.
Re:Skip grep, use awk by Ultimatt on Tuesday July 04, 2017 @08:51AM (#14695551)
Skip awk, use perl....

The alias below sets perl to loop over STDIN splitting each line on more than one whitespace character and populate an array F. The -nE will then Evaluate an expression from the command line looping over the input line-by-line.

    alias glorp='perl -aF"/\s+/" -nE'
So now we have the command `glorp` to play with which has more familiar syntax than awk and all of CPAN available to play with!

    $ [data is generated] | glorp '/Something/ and say $F[2]'
We have access to any Perl module by putting -MModule::Name=function after the command, the following will parse a JSON record per line and glorp out what we wanted:

    $ echo -e '{"hello":"world"}\n{"hello":"cat"}' | glorp 'say decode_json($_)->{hello};' -MJSON=decode_json
    world
    cat
Maybe you are used to using curl too. There is a nice web framework in Perl called Mojolicious (http://mojolicious.org) that provides a convenience module called 'ojo' for command line use. So grabbing the summary sentence from Wikipedia articles is as straight forward as below. Notice Mojolicious lets us use CSS selectors!

    $ echo -e 'grep\nawk\nperl' \
      | glorp 'say g("wikipedia.org/wiki/$F[0]")->dom->at("#mw-content-text > div > p")->all_text' -Mojo
Re:Skip grep, use awk by vidarh on Tuesday July 04, 2017 @09:32AM (#14695741)
Here's the equivalent for Ruby:

    alias glorp='ruby -ane '

    $ [data is generated] | glorp ' ~ /Something/ and puts $F[2]'
Or:

    $ echo -e '{"hello":"world"}\n{"hello":"cat"}' | glorp 'puts JSON.load($_)["hello"] ' -rjson

(Of course Ruby got the -a autosplit-mode and the -n assumed 'while gets(); ... end' loop from Perl along with $_ and $F, so it's very intentional that they're similar)
Re:Skip grep, use awk by kolodny on Tuesday July 04, 2017 @11:12AM (#14696316)
Somewhat related nodejs self plug: Use nip https://github.com/kolodny/nip

    $  echo -e 'this\nis\na\nwhatever foo' | nip 'return /whatever/.test(line) && cols[1]' # foo
Re:Skip grep, use awk by Ultimatt on Tuesday July 04, 2017 @09:40AM (#14695788)
Awesome thanks for sharing this! I was too lazy to give a Ruby example alongside.
Re:Skip grep, use awk by jedisct1 on Tuesday July 04, 2017 @12:35PM (#14696798)
And Ruby regexes are amazing.
Re:Skip grep, use awk by omaranto on Tuesday July 04, 2017 @12:40PM (#14696831)
I thought that they were amazing because they were just like Perl's. Are there any differences?
Re:Skip grep, use awk by leovonl on Tuesday July 04, 2017 @02:31PM (#14697469)
Sorry, but I strongly disagree.

I can think of two very strong reasons to prefer awk over perl:

* busybox implements awk, so its (usually) available even on embedded platforms

* awk syntax is way clearer than perl, anyone that has a few experience in C + shell can figure out what is being done with some googling

awk is also powerful enough to separate output to different files, accumulate inputs, etc. And if you need anything more complex, there are tons of other languages to choose from then (including perl).

Re:Skip grep, use awk by codesnik on Tuesday July 04, 2017 @11:09AM (#14696287)
what's the reason of using -F option? -a defaults " " separator, which already emulates awk behavior. If I'm not wrong, only difference between perl -aF"/\s+/" and perl -a is treating of leading whitespace in lines
Re:Skip grep, use awk by Ultimatt on Tuesday July 04, 2017 @06:55PM (#14698836)
To show you could use other separators defined by any regex you like.
Re:Skip grep, use awk by d33 on Tuesday July 04, 2017 @09:18AM (#14695671)
Hey, it would be awesome to have something like this for Python!
Re:Skip grep, use awk by gmfawcett on Tuesday July 04, 2017 @10:15AM (#14695990)
I wrote a little tool called "pyline" about a hundred years ago... I still reach for it often when I'm in a hurry, and don't have time to futz around with awk/sed.

https://code.activestate.com/recipes/437932-pyline-a-grep-li...

Re:Skip grep, use awk by edraferi on Tuesday July 04, 2017 @11:23PM (#14699793)
That's pretty cool, thanks!
Re:Skip grep, use awk by Ultimatt on Tuesday July 04, 2017 @09:31AM (#14695731)
Not really possible in Python for anything overly useful/complex because of the significant white space, and regular expressions are not a first class object in the language. ipython is the equivalent solution I suppose.

However, Ruby completely inherited this aspect of Perl and is another good candidate. It's just even old systems have a perl installed that will perform the awk like functionality.

Re:Skip grep, use awk by phaemon on Tuesday July 04, 2017 @09:40AM (#14695793)
There is "Pyed Piper" aka `pyp`

https://code.google.com/archive/p/pyp/

"Pyp is a linux command line text manipulation tool similar to awk or sed, but which uses standard python string and list methods as well as custom functions evolved to generate fast results in an intense production environment."

Re:Skip grep, use awk by Ultimatt on Tuesday July 04, 2017 @09:44AM (#14695816)
That's cool, but I feel like you're unlikely to run into it very often in the wild? The vast majority of Linux systems come with ruby/perl out of the box. Plus if Python did support this sort of hackery it would quickly garner most of the bad press Perl has over the years :'(
Re:Skip grep, use awk by bobpaul on Monday July 10, 2017 @07:39PM (#14740532)
Who cares? It's on pypi (https://pypi.python.org/pypi/pyp/2.11) so just `pip install pyp` and then you have it. You're probably not going to want to distribute shell scripts that rely on pyp, but if you're distributing scripts you can just distribute scripts written entirely in Python and use pyp for your own one-liners.
Re:Skip grep, use awk by tyingq on Tuesday July 04, 2017 @09:41AM (#14695805)
The lack of something like Perl/Ruby's -n and -p switches is a barrier as well. Those are handy because they automatically work for either piped input, or files named on the command line.

Of course, you can do all of this in Python, just not as a one liner.

Re:Skip grep, use awk by geophile on Tuesday July 04, 2017 @10:22AM (#14696031)
Re:Skip grep, use awk by prathiks on Tuesday July 04, 2017 @12:14PM (#14696670)
so is the intention to be python equivalent of awk
Re:Skip grep, use awk by ShannonAlther on Tuesday July 04, 2017 @11:34AM (#14696453)
There was an awk/perl thread yesterday too.