News for hackers. Stuff that matters.
Re:Experimental library for scraping websites using OpenAI's... by riku_iki on Saturday March 25, 2023 @05:43PM (#35307746)
its wikidata, not wikipedia, they are two disjoint datasets.
Re:Experimental library for scraping websites using OpenAI's... by ZeroGravitas on Saturday March 25, 2023 @05:44PM (#35307760)
Basically every wikipedia page (across languages) is linked to wikidata, and some infoboxes are generated directly from wikidata, so they're seperate, but overlapping and increasingly so.

https://en.wikipedia.org/wiki/Category:Articles_with_infobox...

edit: slightly wider scope category pointing to pages using wikidata in different ways:

https://en.wikipedia.org/wiki/Category:Wikipedia_categories_...

Re:Experimental library for scraping websites using OpenAI's... by riku_iki on Saturday March 25, 2023 @05:51PM (#35307811)
I agree there is strong overlap between entities, and also infobox values, but both wikidata and wikipedia has many more disjoint datapoints: many tables, factual statements in wikipedia which are not in wikidata, and many statements in wikidata which are not in wikipedia.