Re:Data-Mining Wikipedia for Fun and Profit
by shock-value on Thursday August 19, 2021 @11:03AM
(#28234731)
Generally Wikidata would definitely be the way to go here, though I just now tried to retrace your graph in Wikidata and it seems to be missing at least one relation (Ada of Holland has no children listed -- https://www.wikidata.org/wiki/Q16156475).
[ Parent
| on HN ]
I am doubtful. I tried for a long time to use it to get data or for my taxonomic graph project (https://relatedhow.kodare.com/) and SPARCQL was just not usable at all. The biggest problem was the 60s time limit. Totally not workable for what I wanted. I also had issues with seemingly inconsistent results, but it was hard to tell.
I ended up loading the full nightly db dump and filtering it streaming from the zip instead. Faster and it actually worked.
The code to do that is at https://github.com/boxed/relatedhow
Re:Data-Mining Wikipedia for Fun and Profit
by shock-value on Saturday August 21, 2021 @01:13AM
(#28254405)
There's an alternate Wikidata query engine available here: https://qlever.cs.uni-freiburg.de/wikidata (from https://github.com/ad-freiburg/QLever)
Currently it doesn't support some SPARQL features, but I've found it to generally be quite a bit faster for most queries.