News for hackers. Stuff that matters.
Re:Semantic MediaWiki by detaro on Saturday June 24, 2023 @04:48PM (#36462525)
Would love to hear people's experience with it, especially for more complex things. Seen it used a few times to add consistent metadata to pages (through forms), but never much built on top of such basics.
Re:Semantic MediaWiki by duskwuff on Saturday June 24, 2023 @05:06PM (#36462648)
Perhaps surprisingly, it isn't particularly useful for general encyclopedias like Wikipedia. Real life is messy; it doesn't always reduce cleanly to a set of key/value pairs.

Where extensions like SMW really shine is for wikis which discuss highly structured content, like content in video games. Another extension which I've seen used in this space is Cargo [1].

[1]: https://www.mediawiki.org/wiki/Extension:Cargo

Re:Semantic MediaWiki by resolutebat on Saturday June 24, 2023 @06:08PM (#36463100)
Around 10 years ago I experimented with this to try to create a geographical hierarchy of articles in a wiki.

It was, in a nutshell, not great: there's a fundamental contradiction between wikis, which are very loosely structured and edited by humans who do all sorts of wacky unpredictable shit, and hierarchies, which need to be carefully pruned and maintained.

So this use case didn't work, and I think it's telling that (AFAIK) not a single Wikipedia project has adopted it.

Re:Semantic MediaWiki by detaro on Saturday June 24, 2023 @06:15PM (#36463138)
as duskwuff said, wikipedia projects are a very open scope, that'd be hard to model. So I really wonder if there are any complex (but not as-complex-as-wikipedia) expamples people had success with.
Re:Semantic MediaWiki by bawolff on Saturday June 24, 2023 @10:36PM (#36464685)
Wikimedia adopted Wikibase (aka Wikidata) instead to fill that niche.

SemanticMediaWiki has some questionable scalability when it comes to large sites. Wikimedia was never going to adopt it for technical reasons.

Re:Semantic MediaWiki by Fannon on Sunday June 25, 2023 @02:03AM (#36465469)
I've build a few Semantic MediaWikis in my past and share the concern about scalability. But also be aware that in many cases you might not need such a scalability like managing semantic data for all of Wikimedias major wikis in different languages. For small to medium sized companies or communities, I think SMW can scale well enough - depending on how you use it.

The advantage of SMW imho is still that it brings the semantic / structured data features right into the MediaWiki itself and that there's a big ecosystem of related extensions around it (like Page Forms). So far I don't know of an open-source information management system which comes close in capability to this. Setting this all up and maintaining it is difficult, though - I even had to start developing custom tools to assist me with that once it got more complex.

Re:Semantic MediaWiki by bawolff on Sunday June 25, 2023 @06:50AM (#36466863)
I mean, I'd agree. Most people do not operate at scale, and prematurely optimizing for a scale that most people will never reach is a bad idea.

However if we're talking about Wikipedia, any scaling concerns at all basically make it a non-starter.

Re:Semantic MediaWiki by dankilman on Saturday June 24, 2023 @06:17PM (#36463144)
The guild wars 2 wiki makes great use of it imo
Re:Semantic MediaWiki by zeeZ on Sunday June 25, 2023 @01:20AM (#36465316)
It does, though a recent update to the wiki broke things for some time.

https://wiki.guildwars2.com/wiki/Guild_Wars_2_Wiki:Reporting...

Re:Semantic MediaWiki by RicoElectrico on Saturday June 24, 2023 @06:24PM (#36463180)
Check out WikiChip for an example of it. https://en.wikichip.org/wiki/WikiChip
Re:Semantic MediaWiki by r1ch on Saturday June 24, 2023 @10:50PM (#36464746)
We used SMW at Liquipedia, an esports wiki. The idea was that all the data about players and teams could be exposed through SMW. Unfortunately we're now in the process of transitioning away from it due to major performance issues and data inconsistencies. It might be ok on smaller wikis but for us it's become a source of a lot of problems.
Re:Semantic MediaWiki by Blackthorn on Saturday June 24, 2023 @11:15PM (#36464854)
I used it on a popular wiki for adding metadata that allowed for decently powerful search filterings. It was important to do this because mediawiki's default search is/was absolutely horrible.

We never bothered doing anything like concepts, detailed types, etc...we only used it for basically adding tags to pages that you could query as part of a search. It worked okay with that. We had some issues where we wished a property could be an enumeration or nullable, which I don't think ever happened in one of their many releases.

Adding the metadata represented a bit of a challenge. We eventually had it generated from templates that would usually get filled out anyway.

Re:Semantic MediaWiki by hirundo on Saturday June 24, 2023 @04:50PM (#36462543)
"The purpose of SMW is to allow users to improve the structure and organization of the knowledge in a wiki by adding simple, machine-processable information to wiki articles."

With LLMs those tags have suddenly become a lot cheaper to make. Arguably their cost was the largest barrier to the semantic web vision. But now it may already be practical to semantically markup wikipedia content automatically as submitted. But does AI moot the semantic web, or enable it? Is the semantic layer redundant with an AI interface? I can't predict.

Re:Semantic MediaWiki by ElFitz on Saturday June 24, 2023 @07:12PM (#36463538)
Hard to say.

Depending on the context, read / write model, and how inference costs go, inferring on edit to classify and tag the content could prove more expensive than having users do it client-side on read.

Re:Semantic MediaWiki by Blackthorn on Saturday June 24, 2023 @11:16PM (#36464860)
LLMs ain't gonna be adding trustworthy metadata to pages on my wiki that can be useful for filtering (which is the entire purpose of SMW for us).
Re:Semantic MediaWiki by Fannon on Sunday June 25, 2023 @01:57AM (#36465446)
My guess it that there is some cross-pollination and benefits from and to both sides. Machine Learning / LLMs benefit from having higher quality data that they can be trained from or that you can use to automatically add more relevant context behind the scenes.

Semantic technologies or let's say "structured data entry" can benefit from having better auto-suggestions and validations from machine-learning.

The part that I find real interesting is how we put the users in between, so they're in full control and can use AI as an "assistant", but improve on it. And that the "structured data entry" helps the users to give him structure (that will later be helpful to the end-consumers).

Re:Semantic MediaWiki by nologic01 on Saturday June 24, 2023 @05:11PM (#36462688)
Semantic technologies, linked data never really delivered. The "Semantic" aspect was in sense a hyped, exaggerated term, long before ML/AI hypes became regular abusers of linguistic precision at the service of marketing.

But had they been adopted for what they are: well defined metadata according to vocabularies and ontologies that following accepted, interoperable standards, the development of ground truth and knowledge graphs would have been much more advanced that the current state.

So my guess is that projects like semantic mediawiki, wikibase etc will eventually see renewed roles and gradually link with the more statistical and algorithmic processing of information.

Re:Semantic MediaWiki by simongray on Sunday June 25, 2023 @01:45AM (#36465405)
> But had they been adopted for what they are: well defined metadata according to vocabularies and ontologies that following accepted, interoperable standards, the development of ground truth and knowledge graphs would have been much more advanced that the current state.

Yes, this is very true.

As an addendum, I think many people in industry don't quite realise how prevalent RDF and RDF-based ontologies are in academia. They're one of the main ways researchers integrate data. No one cares about lofty ideals, it's about the usefulness of RDF graph model and the various associated standards built on top of it.

Re:Semantic MediaWiki by la_fayette on Sunday June 25, 2023 @02:18AM (#36465524)
Which fields? Most projects i was involved, web scraping and putting the data in to csv was the main approach...
Re:Semantic MediaWiki by nologic01 on Sunday June 25, 2023 @03:36AM (#36465829)
Biology and biomedical research is the main advanced application I am aware of. Its important enough a set of domains to show there is value in semantic technologies (rdf, owl etc)

I have been wondering why it got adopted there and not elsewhere. Its not clear. Adoption or not of software and information tools is a mix of cultural and economic reasons beyond pure technical aspects.

CSV, json etc are information black holes. Eventually people will realize that the systematic annotation of data within a metadata framework is not fanciful but an enabler of more confident and robust information exchange and processing

[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4652617/

Re:Semantic MediaWiki by simongray on Sunday June 25, 2023 @07:54AM (#36467268)
The key word in my comment is "integrate", not simply producing data. We're using it for language data in my field, e.g. currently I'm integrating the Danish WordNet with the English one + various other datasets.

If you check out the common ontologies in use you will get an idea which areas are covered: https://lov.linkeddata.es/dataset/lov/

Re:Semantic MediaWiki by neilv on Saturday June 24, 2023 @05:24PM (#36462775)
When I was first working on distributed "semantic" Web+HCI+AI stuff, in the late '90s, nobody was as concerned with monetizing -- you just did useful or interesting things, and the money will be figured out eventually.

Today, the first thoughts that comes to my mind aren't what my work could do with this information, but more about sustainability, accuracy, and power.

A bunch of huge companies are going to hoover up all the data you've compiled, and use it in such a way that most people using it will never see your Web site.

So won't be prompted to contribute back wiki-style, won't get all the benefit of your quality control, won't see your donation appeals, won't know you exist, won't see the other things that you would like to see, won't give you analytics on your impact, won't fund you with ad impressions, etc.

Which might all be OK for your use case, but over time it could break most of the current useful percentage of the Web.

Re:Semantic MediaWiki by justinjlynn on Saturday June 24, 2023 @07:52PM (#36463814)
You've described most of open source software development and yet people still write the code because they need the code and want to share the code with others because others shared with them. This isn't the problem you think it is. The problem is that most of the web's current funding model is broken and user hostile. This isn't a problem with semantic data or software development this is a problem with the capitalist model of post-scarcity information production, distribution, and control. That's what needs to change if we're to benefit in proportion to what we contribute instead of those who appropriate and exploit the most benefiting the most. Let the web, as it exists now, break - it hasn't been ours for a long time.
Re:Semantic MediaWiki by neilv on Saturday June 24, 2023 @08:21PM (#36463995)
I'd like to have a better model in place, before the current one breaks.
Re:Semantic MediaWiki by justinjlynn on Sunday June 25, 2023 @01:56AM (#36465441)
If necessity is the mother of invention then revolution is the father.
Re:Semantic MediaWiki by ben_w on Sunday June 25, 2023 @04:01AM (#36465933)
"An Outside Context Problem was the sort of thing most civilisations encountered just once, and which they tended to encounter rather in the same way a sentence encountered a full stop."

— Iain M. Banks, Excession

Despite being fictional, it seems at least as correct as your saying.

Also applies to corporations.

Re:Semantic MediaWiki by justinjlynn on Sunday June 25, 2023 @11:59PM (#36475604)
It is and very much does. Fortunately, there is plausibly little that qualifies as a true outside context problem these days. Authors of fiction, and especially of science fiction, are mostly to thank for that.
Re:Semantic MediaWiki by facu17y on Saturday June 24, 2023 @06:39PM (#36463275)
This project reminds me of this now-ancient blog post by @marcfawzi from the Evolving Trends blog (does not seem to be maintained):

https://evolvingtrends.wordpress.com/2006/06/26/wikipedia-30...

It took more decades and the arrival of Transformers and self-attention (see Attention is All You Need) for the "End of Google" (the search engine, not the company) to become a reality.

I was able to find Marc on Twitter: https://twitter.com/marcfawzi

Seems to have picked up his old idea of a geek-run VC fund, but using AI: https://evolvingtrends.wordpress.com/2006/06/21/geek-run-gee...

Re:Semantic MediaWiki by samuell on Saturday June 24, 2023 @07:08PM (#36463507)
SMW together with templates is pretty fun and inspiring to work with, and you start to think up a lot of cool applications you could do with it.

It is used att places like NASA as a knowledge base [0] and some other places.

I guess what is hindering wide adoption is that it is still quite a bit too much conventions and caveats to learn to become productive with it, that it has a hard time reaching outside niche use cases.

(Basis for speaking my mind on the topic: Developed the RDFIO plugin for RDF import/export in SMW [1])

[0] https://www.semantic-mediawiki.org/wiki/EVA_Wiki

[1] https://www.mediawiki.org/wiki/Extension:RDFIO

Re:Semantic MediaWiki by ak217 on Saturday June 24, 2023 @10:10PM (#36464553)
How does this compare to Wikidata? I've been able to do some powerful things by querying Wikidata (via the Mediawiki query API, not the SPARQL one, which turned out to be unusable). I've also seen Wikipedia editors do great things with templates that I believe use Wikidata as a backend.
Re:Semantic MediaWiki by bawolff on Saturday June 24, 2023 @10:29PM (#36464649)
SemanticMediaWiki is much older than wikidata. Some of the people involved were involved in both.

Big difference is that wikidata is an entirely separate wiki from client wikis, where smw is much more closely integrated into "articles". This allows wikidata to abstract across multiple language wikipedias. In practise that means that wikidata is often used to store factual data outside of articles that than gets pulled in via templates. SMW tends to more get used as a backend to turn mediawiki into a user programmable CRUD/workflow app (particularly with PageForms extension)

SMW is less concerned about scalability and performance. Wikidata puts anything of questionable scalability separate from the system (e.g. sparql api is isolated from rest). SMW would be unlikely to scale to anywhere near wikipedia levels in its current form.

Re:Semantic MediaWiki by lolive on Sunday June 25, 2023 @02:02AM (#36465465)
A proprietary extension for Semantic MediaWiki, called Project Halo, had been developped to extend the ontology management capabilities of the tool.

Still amazed at that video when I rewatch it.

[fyi, the company developing that extension was bought, and Project Halo disappeared in limbo]

https://youtu.be/WXmMHpHjG2o

Re:Semantic MediaWiki by lolive on Sunday June 25, 2023 @02:16AM (#36465515)
Cf Wikipedia, it seems that indeed the support still exists: https://en.m.wikipedia.org/wiki/SMW%2B

Eventually I presume it all goes down to Michael Erdmann [@emwiemaikel on Twitter] having the right to maintain the codebase.

Re:Semantic MediaWiki by lolive on Sunday June 25, 2023 @06:10AM (#36466596)
Nowadays, you would use Obsidian with tags, queries and DataView.

[Or maybe Notion, i never used it]

Re:Semantic MediaWiki by altilunium on Sunday June 25, 2023 @06:47AM (#36466851)
The history behind the development of SMW : https://dl.acm.org/doi/fullHtml/10.1145/3543873.3585579
Re:Semantic MediaWiki by bertrandgorge on Sunday June 25, 2023 @10:37AM (#36468592)
We use it for our project https://3perf.fr (sorry in french. It's a repository of agroecological cases and practices). It's very useful to add some structured metadata to encyclopedical data, in order to add some faceting search capabilities for exemple. Very versatile.