Re:Semantic MediaWiki
by detaro on Saturday June 24, 2023 @04:48PM
(#36462525)
Would love to hear people's experience with it, especially for more complex things. Seen it used a few times to add consistent metadata to pages (through forms), but never much built on top of such basics.
Re:Semantic MediaWiki
by duskwuff on Saturday June 24, 2023 @05:06PM
(#36462648)
Perhaps surprisingly, it isn't particularly useful for general encyclopedias like Wikipedia. Real life is messy; it doesn't always reduce cleanly to a set of key/value pairs.
Where extensions like SMW really shine is for wikis which discuss highly structured content, like content in video games. Another extension which I've seen used in this space is Cargo [1].
Re:Semantic MediaWiki
by resolutebat on Saturday June 24, 2023 @06:08PM
(#36463100)
Around 10 years ago I experimented with this to try to create a geographical hierarchy of articles in a wiki.
It was, in a nutshell, not great: there's a fundamental contradiction between wikis, which are very loosely structured and edited by humans who do all sorts of wacky unpredictable shit, and hierarchies, which need to be carefully pruned and maintained.
So this use case didn't work, and I think it's telling that (AFAIK) not a single Wikipedia project has adopted it.
Re:Semantic MediaWiki
by detaro on Saturday June 24, 2023 @06:15PM
(#36463138)
as duskwuff said, wikipedia projects are a very open scope, that'd be hard to model. So I really wonder if there are any complex (but not as-complex-as-wikipedia) expamples people had success with.
Re:Semantic MediaWiki
by Fannon on Sunday June 25, 2023 @02:03AM
(#36465469)
I've build a few Semantic MediaWikis in my past and share the concern about scalability. But also be aware that in many cases you might not need such a scalability like managing semantic data for all of Wikimedias major wikis in different languages. For small to medium sized companies or communities, I think SMW can scale well enough - depending on how you use it.
The advantage of SMW imho is still that it brings the semantic / structured data features right into the MediaWiki itself and that there's a big ecosystem of related extensions around it (like Page Forms). So far I don't know of an open-source information management system which comes close in capability to this. Setting this all up and maintaining it is difficult, though - I even had to start developing custom tools to assist me with that once it got more complex.
Re:Semantic MediaWiki
by r1ch on Saturday June 24, 2023 @10:50PM
(#36464746)
We used SMW at Liquipedia, an esports wiki. The idea was that all the data about players and teams could be exposed through SMW. Unfortunately we're now in the process of transitioning away from it due to major performance issues and data inconsistencies. It might be ok on smaller wikis but for us it's become a source of a lot of problems.
Re:Semantic MediaWiki
by Blackthorn on Saturday June 24, 2023 @11:15PM
(#36464854)
I used it on a popular wiki for adding metadata that allowed for decently powerful search filterings. It was important to do this because mediawiki's default search is/was absolutely horrible.
We never bothered doing anything like concepts, detailed types, etc...we only used it for basically adding tags to pages that you could query as part of a search. It worked okay with that. We had some issues where we wished a property could be an enumeration or nullable, which I don't think ever happened in one of their many releases.
Adding the metadata represented a bit of a challenge. We eventually had it generated from templates that would usually get filled out anyway.
Re:Semantic MediaWiki
by hirundo on Saturday June 24, 2023 @04:50PM
(#36462543)
"The purpose of SMW is to allow users to improve the structure and organization of the knowledge in a wiki by adding simple, machine-processable information to wiki articles."
With LLMs those tags have suddenly become a lot cheaper to make. Arguably their cost was the largest barrier to the semantic web vision. But now it may already be practical to semantically markup wikipedia content automatically as submitted. But does AI moot the semantic web, or enable it? Is the semantic layer redundant with an AI interface? I can't predict.
Re:Semantic MediaWiki
by ElFitz on Saturday June 24, 2023 @07:12PM
(#36463538)
Hard to say.
Depending on the context, read / write model, and how inference costs go, inferring on edit to classify and tag the content could prove more expensive than having users do it client-side on read.
Re:Semantic MediaWiki
by Fannon on Sunday June 25, 2023 @01:57AM
(#36465446)
My guess it that there is some cross-pollination and benefits from and to both sides. Machine Learning / LLMs benefit from having higher quality data that they can be trained from or that you can use to automatically add more relevant context behind the scenes.
Semantic technologies or let's say "structured data entry" can benefit from having better auto-suggestions and validations from machine-learning.
The part that I find real interesting is how we put the users in between, so they're in full control and can use AI as an "assistant", but improve on it. And that the "structured data entry" helps the users to give him structure (that will later be helpful to the end-consumers).
Re:Semantic MediaWiki
by nologic01 on Saturday June 24, 2023 @05:11PM
(#36462688)
Semantic technologies, linked data never really delivered. The "Semantic" aspect was in sense a hyped, exaggerated term, long before ML/AI hypes became regular abusers of linguistic precision at the service of marketing.
But had they been adopted for what they are: well defined metadata according to vocabularies and ontologies that following accepted, interoperable standards, the development of ground truth and knowledge graphs would have been much more advanced that the current state.
So my guess is that projects like semantic mediawiki, wikibase etc will eventually see renewed roles and gradually link with the more statistical and algorithmic processing of information.
Re:Semantic MediaWiki
by simongray on Sunday June 25, 2023 @01:45AM
(#36465405)
> But had they been adopted for what they are: well defined metadata according to vocabularies and ontologies that following accepted, interoperable standards, the development of ground truth and knowledge graphs would have been much more advanced that the current state.
Yes, this is very true.
As an addendum, I think many people in industry don't quite realise how prevalent RDF and RDF-based ontologies are in academia. They're one of the main ways researchers integrate data. No one cares about lofty ideals, it's about the usefulness of RDF graph model and the various associated standards built on top of it.
Re:Semantic MediaWiki
by nologic01 on Sunday June 25, 2023 @03:36AM
(#36465829)
Biology and biomedical research is the main advanced application I am aware of. Its important enough a set of domains to show there is value in semantic technologies (rdf, owl etc)
I have been wondering why it got adopted there and not elsewhere. Its not clear. Adoption or not of software and information tools is a mix of cultural and economic reasons beyond pure technical aspects.
CSV, json etc are information black holes. Eventually people will realize that the systematic annotation of data within a metadata framework is not fanciful but an enabler of more confident and robust information exchange and processing
Re:Semantic MediaWiki
by simongray on Sunday June 25, 2023 @07:54AM
(#36467268)
The key word in my comment is "integrate", not simply producing data. We're using it for language data in my field, e.g. currently I'm integrating the Danish WordNet with the English one + various other datasets.
Re:Semantic MediaWiki
by neilv on Saturday June 24, 2023 @05:24PM
(#36462775)
When I was first working on distributed "semantic" Web+HCI+AI stuff, in the late '90s, nobody was as concerned with monetizing -- you just did useful or interesting things, and the money will be figured out eventually.
Today, the first thoughts that comes to my mind aren't what my work could do with this information, but more about sustainability, accuracy, and power.
A bunch of huge companies are going to hoover up all the data you've compiled, and use it in such a way that most people using it will never see your Web site.
So won't be prompted to contribute back wiki-style, won't get all the benefit of your quality control, won't see your donation appeals, won't know you exist, won't see the other things that you would like to see, won't give you analytics on your impact, won't fund you with ad impressions, etc.
Which might all be OK for your use case, but over time it could break most of the current useful percentage of the Web.
Re:Semantic MediaWiki
by justinjlynn on Saturday June 24, 2023 @07:52PM
(#36463814)
You've described most of open source software development and yet people still write the code because they need the code and want to share the code with others because others shared with them. This isn't the problem you think it is. The problem is that most of the web's current funding model is broken and user hostile. This isn't a problem with semantic data or software development this is a problem with the capitalist model of post-scarcity information production, distribution, and control. That's what needs to change if we're to benefit in proportion to what we contribute instead of those who appropriate and exploit the most benefiting the most. Let the web, as it exists now, break - it hasn't been ours for a long time.
Re:Semantic MediaWiki
by ben_w on Sunday June 25, 2023 @04:01AM
(#36465933)
"An Outside Context Problem was the sort of thing most civilisations encountered just once, and which they tended to encounter rather in the same way a sentence encountered a full stop."
— Iain M. Banks, Excession
Despite being fictional, it seems at least as correct as your saying.
Re:Semantic MediaWiki
by justinjlynn on Sunday June 25, 2023 @11:59PM
(#36475604)
It is and very much does. Fortunately, there is plausibly little that qualifies as a true outside context problem these days. Authors of fiction, and especially of science fiction, are mostly to thank for that.
It took more decades and the arrival of Transformers and self-attention (see Attention is All You Need) for the "End of Google" (the search engine, not the company) to become a reality.
Re:Semantic MediaWiki
by samuell on Saturday June 24, 2023 @07:08PM
(#36463507)
SMW together with templates is pretty fun and inspiring to work with, and you start to think up a lot of cool applications you could do with it.
It is used att places like NASA as a knowledge base [0] and some other places.
I guess what is hindering wide adoption is that it is still quite a bit too much conventions and caveats to learn to become productive with it, that it has a hard time reaching outside niche use cases.
(Basis for speaking my mind on the topic: Developed the RDFIO plugin for RDF import/export in SMW [1])
Re:Semantic MediaWiki
by ak217 on Saturday June 24, 2023 @10:10PM
(#36464553)
How does this compare to Wikidata? I've been able to do some powerful things by querying Wikidata (via the Mediawiki query API, not the SPARQL one, which turned out to be unusable). I've also seen Wikipedia editors do great things with templates that I believe use Wikidata as a backend.
Re:Semantic MediaWiki
by bawolff on Saturday June 24, 2023 @10:29PM
(#36464649)
SemanticMediaWiki is much older than wikidata. Some of the people involved were involved in both.
Big difference is that wikidata is an entirely separate wiki from client wikis, where smw is much more closely integrated into "articles". This allows wikidata to abstract across multiple language wikipedias. In practise that means that wikidata is often used to store factual data outside of articles that than gets pulled in via templates. SMW tends to more get used as a backend to turn mediawiki into a user programmable CRUD/workflow app (particularly with PageForms extension)
SMW is less concerned about scalability and performance. Wikidata puts anything of questionable scalability separate from the system (e.g. sparql api is isolated from rest). SMW would be unlikely to scale to anywhere near wikipedia levels in its current form.
We use it for our project https://3perf.fr (sorry in french. It's a repository of agroecological cases and practices). It's very useful to add some structured metadata to encyclopedical data, in order to add some faceting search capabilities for exemple. Very versatile.
Where extensions like SMW really shine is for wikis which discuss highly structured content, like content in video games. Another extension which I've seen used in this space is Cargo [1].
[1]: https://www.mediawiki.org/wiki/Extension:Cargo
It was, in a nutshell, not great: there's a fundamental contradiction between wikis, which are very loosely structured and edited by humans who do all sorts of wacky unpredictable shit, and hierarchies, which need to be carefully pruned and maintained.
So this use case didn't work, and I think it's telling that (AFAIK) not a single Wikipedia project has adopted it.
SemanticMediaWiki has some questionable scalability when it comes to large sites. Wikimedia was never going to adopt it for technical reasons.
The advantage of SMW imho is still that it brings the semantic / structured data features right into the MediaWiki itself and that there's a big ecosystem of related extensions around it (like Page Forms). So far I don't know of an open-source information management system which comes close in capability to this. Setting this all up and maintaining it is difficult, though - I even had to start developing custom tools to assist me with that once it got more complex.
However if we're talking about Wikipedia, any scaling concerns at all basically make it a non-starter.
https://wiki.guildwars2.com/wiki/Guild_Wars_2_Wiki:Reporting...
We never bothered doing anything like concepts, detailed types, etc...we only used it for basically adding tags to pages that you could query as part of a search. It worked okay with that. We had some issues where we wished a property could be an enumeration or nullable, which I don't think ever happened in one of their many releases.
Adding the metadata represented a bit of a challenge. We eventually had it generated from templates that would usually get filled out anyway.
With LLMs those tags have suddenly become a lot cheaper to make. Arguably their cost was the largest barrier to the semantic web vision. But now it may already be practical to semantically markup wikipedia content automatically as submitted. But does AI moot the semantic web, or enable it? Is the semantic layer redundant with an AI interface? I can't predict.
Depending on the context, read / write model, and how inference costs go, inferring on edit to classify and tag the content could prove more expensive than having users do it client-side on read.
Semantic technologies or let's say "structured data entry" can benefit from having better auto-suggestions and validations from machine-learning.
The part that I find real interesting is how we put the users in between, so they're in full control and can use AI as an "assistant", but improve on it. And that the "structured data entry" helps the users to give him structure (that will later be helpful to the end-consumers).
But had they been adopted for what they are: well defined metadata according to vocabularies and ontologies that following accepted, interoperable standards, the development of ground truth and knowledge graphs would have been much more advanced that the current state.
So my guess is that projects like semantic mediawiki, wikibase etc will eventually see renewed roles and gradually link with the more statistical and algorithmic processing of information.
Yes, this is very true.
As an addendum, I think many people in industry don't quite realise how prevalent RDF and RDF-based ontologies are in academia. They're one of the main ways researchers integrate data. No one cares about lofty ideals, it's about the usefulness of RDF graph model and the various associated standards built on top of it.
I have been wondering why it got adopted there and not elsewhere. Its not clear. Adoption or not of software and information tools is a mix of cultural and economic reasons beyond pure technical aspects.
CSV, json etc are information black holes. Eventually people will realize that the systematic annotation of data within a metadata framework is not fanciful but an enabler of more confident and robust information exchange and processing
[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4652617/
If you check out the common ontologies in use you will get an idea which areas are covered: https://lov.linkeddata.es/dataset/lov/
Today, the first thoughts that comes to my mind aren't what my work could do with this information, but more about sustainability, accuracy, and power.
A bunch of huge companies are going to hoover up all the data you've compiled, and use it in such a way that most people using it will never see your Web site.
So won't be prompted to contribute back wiki-style, won't get all the benefit of your quality control, won't see your donation appeals, won't know you exist, won't see the other things that you would like to see, won't give you analytics on your impact, won't fund you with ad impressions, etc.
Which might all be OK for your use case, but over time it could break most of the current useful percentage of the Web.
— Iain M. Banks, Excession
Despite being fictional, it seems at least as correct as your saying.
Also applies to corporations.
https://evolvingtrends.wordpress.com/2006/06/26/wikipedia-30...
It took more decades and the arrival of Transformers and self-attention (see Attention is All You Need) for the "End of Google" (the search engine, not the company) to become a reality.
I was able to find Marc on Twitter: https://twitter.com/marcfawzi
Seems to have picked up his old idea of a geek-run VC fund, but using AI: https://evolvingtrends.wordpress.com/2006/06/21/geek-run-gee...
It is used att places like NASA as a knowledge base [0] and some other places.
I guess what is hindering wide adoption is that it is still quite a bit too much conventions and caveats to learn to become productive with it, that it has a hard time reaching outside niche use cases.
(Basis for speaking my mind on the topic: Developed the RDFIO plugin for RDF import/export in SMW [1])
[0] https://www.semantic-mediawiki.org/wiki/EVA_Wiki
[1] https://www.mediawiki.org/wiki/Extension:RDFIO
Big difference is that wikidata is an entirely separate wiki from client wikis, where smw is much more closely integrated into "articles". This allows wikidata to abstract across multiple language wikipedias. In practise that means that wikidata is often used to store factual data outside of articles that than gets pulled in via templates. SMW tends to more get used as a backend to turn mediawiki into a user programmable CRUD/workflow app (particularly with PageForms extension)
SMW is less concerned about scalability and performance. Wikidata puts anything of questionable scalability separate from the system (e.g. sparql api is isolated from rest). SMW would be unlikely to scale to anywhere near wikipedia levels in its current form.
Still amazed at that video when I rewatch it.
[fyi, the company developing that extension was bought, and Project Halo disappeared in limbo]
https://youtu.be/WXmMHpHjG2o
Eventually I presume it all goes down to Michael Erdmann [@emwiemaikel on Twitter] having the right to maintain the codebase.
[Or maybe Notion, i never used it]