It also seems Wikimedia isn't trying to relicense the content in any way that strips its e.g. CC-SA status, but rather providing the licenses as context alongside each API call. https://helpcenter.enterprise.wikimedia.com/hc/en-us/article...

It's worth noting that https://creativecommons.org/faq/#artificial-intelligence-and... itself takes the general stance that "as a general matter text and data mining in the United States is considered a fair use and does not require permission under copyright."

But as a practical matter, I wouldn't be surprised if some Wikipedia editors balk at their volunteer work being actively marketed and reformatted for ease of LLM training by the very platform that solicited their volunteer services, regardless of their works' legal status and Wikimedia's technical respect of that legal status.

> I wouldn't be surprised if some Wikipedia editors balk at their volunteer work being actively marketed and reformatted for ease of LLM training

As someone who avidly edited Wikipedia for 6-8 years, I am happy to see my volunteer work used for LLM training. I also agree some other editors likely aren't.

Given that all Wikipedia editors have explicitly consented to their content being released under the Creative Commons Attribution-ShareAlike 4.0 License, they don't get a choice about their content being used for any purpose.

Redistribution of content is an entirely different matter, and the legal status of copyrighted material in relation to LLM training is an open issue that is currently the subject of litigation.

Wikimedia Foundation’s perspective on this [1]:

> "it is important to note that Creative Commons licenses allow for free reproduction and reuse, so AI programs like ChatGPT might copy text from a Wikipedia article or an image from Wikimedia Commons. However, it is not clear yet whether massively copying content from these sources may result in a violation of the Creative Commons license if attribution is not granted. Overall, it is more likely than not if current precedent holds that training systems on copyrighted data will be covered by fair use in the United States, but there is significant uncertainty at time of writing."

The new Wikimedia Enterprise APIs facilitate attribution. For example, the "api.enterprise.wikimedia.com/v2/structured-contents/{name}" response [2] includes an "editor" object in a "version" object. So the Wikipedia editor who most recently edited the article seems quite feasible to attribute. ML apps could incorporate such attribution in their offering, and help satisfy the "BY" clause in the underlying CC-BY-SA 4.0 license for Wikipedia content.

---

1. https://meta.wikimedia.org/wiki/Wikilegal/Copyright_Analysis...

2. https://enterprise.wikimedia.com/docs/on-demand/#article-str...

As another editor, I think they might be a vocal minority. :)

> But as a practical matter, I wouldn't be surprised if some Wikipedia editors balk at their volunteer work being actively marketed and reformatted for ease of LLM training by the very platform that solicited their volunteer services,

I think that will heavily depend on just what the money goes to.

A better user experience, tightening up the code behind things, fewer nag screens for donations? Justifiable.

Jimmy Wales going from decently-compensated to a bad case of Founder Syndrome? Not so justifiable.

Founder Syndrome: a psychological condition wherein a person who starts a venture believes the venture should earn them a net worth on the order of billions of dollars, regardless of its actual economic value, and is willing to seriously enshittify the venture's product or service in order to make this delusion come true, especially in preparation for an IPO. See also: /u/spez, SPAC, Facebook, Unity Engine

> Jimmy Wales going from decently-compensated to a bad case of Founder Syndrome? Not so justifiable.

Jimmy wales is not paid at all (he has a board seat, but that doesn't come with any money)

He of course has leveraged his fame from being "founder" quite extensively. I think most of his money comes from fandom which he is also one of the founders of.

I don't have a problem with having paid special services, but the "machine parsable formats" is a bit troubling since I think that should be a core part of the open wikipedia project.

I submit this link after coming across this site while Googling for info on parsing wikipedia "infoboxes". I plan to check out their "Article Structured Contents (BETA)" API. Improving infoboxes to be machine-readable seems important. And it would be bad if didn't do this because it's a revenue stream for them.

DBPedia has been providing machine-readable info drawn from Wikipedia infoboxes for a long time, and that's absolutely an open project. But there are drawbacks to that approach, nowadays Wikipedia users are broadly expected to input that info into Wikidata (a language-independent project, providing its data under the CC0 license) and the Wikipedias are gradually moving towards fetching the information from there as well. The English Wikipedia is somewhat behind on that, so DBPedia might still be useful for parsing info from it.

There’s also Wikidata which has machine readable data for everyone

https://m.wikidata.org/wiki/Wikidata:Main_Page

> Wikidata is a free and open knowledge base that can be read and edited by both humans and machines.

> Wikidata acts as central storage for the structured data of its Wikimedia sister projects including Wikipedia, Wikivoyage, Wiktionary, Wikisource, and others.

Yes, but it is different dataset, many(majority?) infoboxes are not in wikidata

Just fyi, i think the plan for machine readable infoboxes is stick the data in wikidata, which has its own dumps in machine readable format as well as a publicly accessible SPARQL query endpoint.

Its actually quite cool. If you have never played with it i encourage checking out some of the example queries (there is a button labelled examples) on https://query.wikidata.org/

Machine-readable infoboxes are surprisingly painful because an "infobox" isn't a top-level thing in mediawiki. Rather, it's something that emerged from community practice -- a template that's slightly-standardized, and has a sprawling network of subtemplates for specific categories of article. Then the assorted other language wikipedias have their own variants of these templates with their own names, etc.

Pulling it into a mediawiki extension (or core) and making it part of the page-level metadata gets suggested pretty frequently, but it's a bit contentious amongst the hardcore editors who'd need to actually adopt such things. The template-based nature of the current infoboxes mean that they're very accessible to the community, and it's easy to spin off new variants or make changes without getting programmers to help you.

There's slow movement towards getting the sort of data that winds up in infoboxes into wikidata, but it's still somewhat spotty.

(If you've never done it, it can be quite edifying to install mediawiki for yourself and seeing how much of the surrounding infrastructure of wikipedia is absent because it's all templates.)

I once tried to export a simple infobox template with dependencies and it exported 30-40 different template/module pages.

CC-BY-SA to be specific. How does attribution of derivative works work in LLMs (etc)? When is the content they produce required to be CC-BY-SA as well (by virtue of the -SA part of the license)?

LLM output can't be copywritten as it's not the work of a human. The company would presumably have to include attribution to every piece of data the system is trained on for every query? Seems absurd, but then how would attribution work? Not really considered when the license was written.

Licenses go beyond copyright though, they tell you what you can and can't do with something someone else made. They're contracts.

I'm sure we'll be seeing more lawsuits (and likely new regulation and licenses) around this.

Are you saying LLMs are not derivatives of their source material? Why not?

I agree on lawsuits being likely and needed to establish precedent. CC-BY-SA specifies "other rights such as publicity, privacy, or moral rights may limit how you use the material".

At some point, maybe poorly-trained chatbots that consistently produce what's seen as avoidably/negligantly poor results may become regulated. Like how if a company poorly trains its employees, it is on the hook for their employees' behavior.

> machine parsable formats,

Isn't the point of wikidata to offer machine-parseable formats?

Also: how can those formats be locked into a paywall?

wikidata is not the same as wikipedia infoboxes.

wikipedia infoboxes API is already paywalled, pricing starts from $0.01/req.

What I am proposing is that Wikipedia infoboxes should be included in Wikidata

this is good proposal, but it is far from what we have and not clear if it will happen.

But now unfortunately the Wikimedia foundation has a motive to block such a proposal, in order to not threaten their revenue stream

And that's the problem with making money off convenience. If the free thing becomes better, then it cannibalizes the paid stuff. So if you take this approach to funding, you will want the free tier to be perpetually bad as much as possible.

I think sorting current infoboxes mess is not trivial problem. We will see what will be quality of their solution, it could be lower than actual wikidata.