News for hackers. Stuff that matters.

A comment on Extracting data from Wikipedia using curl, grep, cut and other bash commands.

Re:Extracting data from Wikipedia using curl, grep, cut and ... by yarrel on Monday August 15, 2016 @04:58PM (#12293379)
A couple of years ago I found Perl was fastest at processing Wikipedia dumps.

It also didn't require having a JVM preloaded to make startup times acceptable during development (naming no other tools).

I do use shell tools to process data, a lot. They're particularly good for exploratory programming and initial analysis of new datasets.

[ Parent | on HN ]
Re:Extracting data from Wikipedia using curl, grep, cut and ... by Steeeve on Monday August 15, 2016 @11:39PM (#12295383)
cut, awk, grep, and perl can churn through an initial data dump like nobody's business.
[ Parent | on HN ]