News for hackers. Stuff that matters.
Re:Wikipedia is swimming in money–why is it begging people t... by bawolff on Monday May 31, 2021 @03:10AM (#27341149)
More like, we put the facts and knowledge in a central (mysql) db, so that instead of updating them in 300 places (Wikipedia has a lot of languages), you just update them in one place.

And since the data happened to be there anyways, we added an RDF export feature, and threw that into a BlazeGraph DB so that people could do SPARQL queries if they want. The (not really federated) SPARQL queries have basically nothing to do with the central-place-to-store-facts that are used in multiple places, feature, but it was pretty cheap to do once everything was in a central place.

Re:Wikipedia is swimming in money–why is it begging people t... by lifeisstillgood on Monday May 31, 2021 @03:50AM (#27341383)
Is there a entry point for "how to make FOSS contributions to wiki-projects". I at the very least would be interested in finding out how all the pieces fit together, and maybe browsing the big list to see if anything was in my wheelhouse.

Edit: And thank you for some insight into the back end of wikipedia - I did not know multiple languages was such a big push and it is such a sensible direction . Keep up the good work and keep pestering me for cash each year.

Re:Wikipedia is swimming in money–why is it begging people t... by bawolff on Monday May 31, 2021 @04:03AM (#27341466)
https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hac... is probably what you are looking for.

There's a very old architecture doc from 2012: https://www.mediawiki.org/wiki/Manual:MediaWiki_architecture its kind of outdated and missing a lot of the newer pieces.

p.s. I used to work for the Wikimedia foundation about 1.5 years ago, but don't anymore. Prior to working there I was a volunteer for the FOSS project, and in theory I still am but I honestly don't do much anymore. My opinions are my own and don't represent WMF or anyone else.

Re:Wikipedia is swimming in money–why is it begging people t... by j-pb on Monday May 31, 2021 @04:08AM (#27341497)
As if thowing the messy complexities of MySQL queries into the serene simplicity of static hypertext is somehow better.

How often is numerical (which is the only knowledge you can really automate this way) knowledge updated? How often is said knowledge dependent on cultural idiosyncracies? How much effort is the code maintenance of the specialised editing system? How much effort is maintainance of the centralised infrastructure. How much effort is it really to change the data in multiple places, given that enough eyes every bug is shallow? What are the oppirtunity costs of not being able to properly distribute and share articles because they are suddenly tied to a very specific codebase? How much infrastructure cost do I have because I can't get volunteers to host my data?

Overengineering solving the wrong problems at it's finest, no wonder the software has stayed this bad for decades now.

Re:Wikipedia is swimming in money–why is it begging people t... by bawolff on Monday May 31, 2021 @04:26AM (#27341619)
> How often is numerical (which is the only knowledge you can really automate this way) knowledge updated?

Numerical knowledge is not the only knowledge that is stored there.

I'm going to go with update rate of often. It has a higher edit rate than english wikipedia does.

> How often is said knowledge dependent on cultural idiosyncracies?

Sometimes that is true, other times it isn't. The system is used where it makes sense, and not used where it doesn't.

> How much effort is the code maintenance of the specialised editing system? How much effort is maintainance of the centralised infrastructure.

There is certainly some. I wouldn't say its dominating, but it certainly exists. There is plenty of centralized infrastructure for other things too.

Prior to the introduction of wikidata, more complex fragile systems existed to try and keep things in sync in a more manual fashion. They also had a cost.

> How much effort is it really to change the data in multiple places, given that enough eyes every bug is shallow?

A lot. And we don't have that many eyes in lots of languages. There's a lot after all: https://en.wikipedia.org/wiki/Special:SiteMatrix

> What are the oppirtunity costs of not being able to properly distribute and share articles because they are suddenly tied to a very specific codebase

Not sure what this has to do with it. Articles are written in a custom markup language. That's already tying it to a specific code base quite heavily. The wikidata dependency is pretty trivial, and there are open apis to get info out of wikidata, and regularly made dumps of all the data in wikidata if you want to copy it. If that's not good enough, just copy the rendered version instead of the source.

> How much infrastructure cost do I have because I can't get volunteers to host my data

I'm not sure what you're talking about. Wikipedia was never hosted in some magical peer-to-peer fashion on volunteer web servers, nor is there any situation where it is likely to be hosted that way. The goals of (near) instant update time, as well as some central quality control and user blocking, makes distributed hosting pretty impossible.

Re:Wikipedia is swimming in money–why is it begging people t... by j-pb on Monday May 31, 2021 @05:00AM (#27341876)
> I'm not sure what you're talking about. Wikipedia was never hosted in some magical peer-to-peer fashion on volunteer web servers, nor is there any situation where it is likely to be hosted that way. The goals of (near) instant update time, as well as some central quality control and user blocking, makes distributed hosting pretty impossible.

Everything wrong with wikipedia summed up in one paragraph, nice!

Re:Wikipedia is swimming in money–why is it begging people t... by bawolff on Monday May 31, 2021 @05:17AM (#27342011)
I mean, if you want to make a website with different goals and properties than wikipedia, by all means go make one. Wikipedia is just one way of doing things. Its hardly the last word on how to make an online encyclopedia, and quite frankly, some competition in this space would probably be good for everyone.

But disagreing with wikipedia's goals as a product is very different from a claim that the solution intended to meet those product goals is over-engineered.

Re:Wikipedia is swimming in money–why is it begging people t... by j-pb on Monday May 31, 2021 @06:36AM (#27342549)
> Wikipedia's purpose is to benefit readers by acting as a free widely accessible encyclopedia;

How are centralization, real time editing, and user blocking properties toward the above goal?

You probably know full well that network effects are prohibiting any other collaborative knowledge base from from ever croping up again.

All big companies link against wikipedia because the network effect, all funding and donations go to wikipedia due to the network effect, all editors spend their time on wikipedia due to the network effect.

Wikipedia is eating it's own childen, and if we ever want to get something better, it either needs to grow up and start acting like a responsible adult (which we know it won't because at it's core is the failed web equivalent of the stanford prison expetiment), or die and make space for a new genration of tools.