News for hackers. Stuff that matters.

A comment on How Akka Streams can be used to process the Wikidata dump in parallel. Parent comment

Re:How Akka Streams can be used to process the Wikidata dump... by mhuffman on Tuesday June 23, 2015 @10:44AM (#9764918)
How about ~17 minutes (including wikipedia data download and extraction time)! Using json-wikipedia and lbzip2

[1] https://github.com/diegoceccarelli/json-wikipedia

[ Parent | on HN ]
Re:How Akka Streams can be used to process the Wikidata dump... by mtrn on Tuesday June 23, 2015 @11:00AM (#9765036)
Thanks, JSON exports make wikipedia data much more approachable.
[ Parent | on HN ]