News for hackers. Stuff that matters.

valexiev

Re:Getting bitten by Intel's poor naming schemes on Friday December 19, 2025 @04:21AM (#46323840)
Sounds like a great candidate for a Cybersecurity Knowledge Graph.
Re:Wikidata or Scraping Wikipedia on Tuesday August 31, 2021 @06:20AM (#28366018)
I have no problems with the data model, but sadly you can't insert RDF statements: you have to go through tools like QS and wikidata-cli and the WD update performance is dismal.

See https://phabricator.wikimedia.org/T290061, which I posted in https://phabricator.wikimedia.org/project/board/5504/ for DataQualityDays2021

Re:Wikidata or Scraping Wikipedia on Tuesday August 31, 2021 @05:42AM (#28365770)
Scientific Articles are in a similar situation: when importing one from a bibliography database, you won't always find every author... So people made an alternative prop "author name" and some disambiguation tools that allow users to gradually replace those with "author" links to real persons.

Let's put aside the question whether each song ever written should be in WD: I believe all of this data, modeled more elaborately, is available on MusicBrainz. There's a difference between a work (eg "Both Sides Now") and its particular rendition in an album (as you said that track is "a cover"), and MusicBrainz makes that distinction and captures both, but I think WD doesn't (I don't work on music in WD, so I haven't checked).

If you really want all this data in WD then I guess you could import it from MusicBrainz... a massive undertaking.

> Wikidata's own parser (however it works)

There's no such thing (in contrast, DBpedia has the dbpedia extraction framework, which is fairly good but not perfect and suffers greatly from the various ways people use to describe the same thing). WD has tools like QS and wikibase-cli, and people write bots to scrape and contribute specific kinds of data.

BTW the Wikipedia link https://en.wikipedia.org/wiki/Both_Sides,_Now#Other_recordin... is wrong (invalid): you can't have two anchors in a link.

Cheers!

Re:Wikidata or Scraping Wikipedia on Tuesday August 31, 2021 @05:33AM (#28365708)
IMHO WD SPARQL should reject invalid literals: https://phabricator.wikimedia.org/T253718
Re:Wikidata or Scraping Wikipedia on Tuesday August 31, 2021 @05:22AM (#28365610)
The WD SPARQL editor has auto-complete (eg type "wdt:award" and press control-space) and readout on hover.

To make the query more readable, use some comments (see my query above).

Yes, WD SPARQL has a firm timeout of 1 minute then may even cut out the response in half. I think it's falling victim of its own popularity (the API is imho much less popular).

There are optimization techniques that one can use, but they take some experience and patience. One good way is to use federated SPARQL insert to a local repo (assuming you want to selectively copy and reshape RDF data), eg our GraphDB repo has batching of federated queries that avoids the timeout.

Re:Wikidata or Scraping Wikipedia on Tuesday August 31, 2021 @05:17AM (#28365585)
?awardYear and ?dateOfBirth are literals, so you don't need to take *Label of them (that's only useful for Qnnn nodes).

Below I use a blank node (since you don't need the URL of ?statement) to simplify the query, and calculate the age as a difference of the two years:

    SELECT ?awardYear ?age ?winnerLabel WHERE {
      SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
      ?winner p:P166 [ # award won
          ps:P166 wd:Q185667; # Turing award
          pq:P585 ?awardDate]; # point in time
        wdt:P569 ?birthDate.
      bind(year(?awardDate) as ?awardYear)
      bind(?awardYear-year(?birthDate) as ?age)
    }
    ORDER BY ?age