News for hackers. Stuff that matters.

A comment on Extracting data from Wikipedia using curl, grep, cut and other bash commands.

Re:Extracting data from Wikipedia using curl, grep, cut and ... by ShakataGaNai on Monday August 15, 2016 @07:11PM (#12294227)
errrrrrrrrk. Extracting raw wiki-markup and trying to use it? Not the greatest of idea. The only true parser of that language is mediawiki. Doing it yourself is a recipe for a massive headache.
[ Parent | on HN ]
Re:Extracting data from Wikipedia using curl, grep, cut and ... by ClayFerguson on Monday August 15, 2016 @10:29PM (#12295126)
I was thinking the same thing. Wikimedia makes all of wikipedia available for download. You don't need to screen scrape. LOL. I guess they had some other wikis they want to get data from but the "main" worldwide wikipedia site, that everyone thinks of as wikipedia makes the data freely downloadable, and I've downloaded it before.
[ Parent | on HN ]