nickvincent
IMO: Renegotiating for "public approval" will always be something computing organizations have to engage with when making advances in tech that's downstream of publicly created records, knowledge, etc.
(That being said, I personally think this announcement is overall a win for healthier data flow, in particular because these kind of deals make the value of data explicit.)
Plus additional commentary here: https://twitter.com/nickmvincent/status/1623409493584519168 (in short: I think this situation is comparable to a "Trap Street" https://en.wikipedia.org/wiki/Trap_street that reveals when a map seller copies another cartographer)
I hadn't seen the Twitch plays pokemon hypothesis though (from another comment here), I wonder if it could be both!
Thus, I honestly think firms operating generative AI should be walking on eggshells to avoid placing blame on "open-source". Rather, they really should going out of their way to channel as much positive energy towards it as possible.
Still, agree the charitable interpretation is that this just purely descriptive.
If you're an artist, of course you can make changes to your process or your content that modify how it's used. To argue otherwise is analogous to arguing we must always be mindful to stay in the surveillance cameras' view when walking about the streets.
Yes, it's a never-ending back-and-forth game that the obfuscater will probably lose in the long run (though abstractly, an obfuscation technique with >50% adoption could "win" long-term) . And yes, it's important to stay apprised about how effective such tools are.
But in the short term, the existence of these tools provides a critical counter-measure to the current narrative, which is basically that everything that can be scraped will be scraped. Returning to the cameras & streets analogy, obfuscation tools are maps that tell us about routes out of view of the cameras (even though these routes may often be blocked off or inconvenient).
Whether you hate AI art or love it, I honestly believe both sides can get behind understanding obfuscation and poisoning and making tools available: those opposed will use the tools, those who want to improve generative AI can learn from the counter-measures, etc. This kind of thing can be part of a healthy deliberative process around these emerging technologies.
However, I'm not aware of anyone actively trying to adapt these techniques for large generative AI models (though would be great to see).
As far as I know, nobody is even thinking about doing the very expensive experiments needed to get ground truth data for formal attribution techniques in the generative AI context (for a given prompt, retrain your model so you can see how the output changes when a particular training example or group of examples is omitted or added), so we're nowhere near building true attribution systems for these very large models. Centering the training data will be net good for public discourse on the topic.
That said, I see why people want to push back on some of the language used here.
I should've been more specific: I was thinking mainly of the artists v. stable diffusion lawsuit which makes the specific technical claim that the stable diffusion software (which includes a bunch of "weights files") includes compressed copies of the training data. (Line 17, "By training Stable Diffusion on the Training Images, Stability caused those images to be stored at and incorporated into Stable Diffusion as compressed copies", https://stablediffusionlitigation.com/pdf/00201/1-1-stable-d...).
I expect that if the decision hinges on this claim, that could have far reaching implications re: model licensing. I think this along the lines of what you've laid out here!
Not a lawyer, but as I understand the most likely way this question will be answered (for practical purposes in the US) is via the ongoing lawsuits against GitHub Copilot and Stable Diffusion and Midjourney.
I personally agree the creativity is in the source images and the training code, but think that unless it is decided that for legal purposes "AI Artifacts" (the files containing model weights, embedding, etc.) are just transformations of training data and therefore content and subject to the same legal standards as content, I see a lot of value in trying to let people license training and code and models separately. And if models are just transformations of content, I expect we can adjust the norms around licensing to achieve similar outcomes (i.e., trying to balance open sharing with some degree of creator-defined use restriction).
See this FAQ here: https://www.licenses.ai/faq-2
Specifically:
Q: "Are OpenRAILs considered open source licenses according to the Open Source Definition? NO."
A: "THESE ARE NOT OPEN SOURCE LICENSES, based on the definition used by Open Source Initiative, because it has some restrictions on the use of the licensed AI artifact.
That said, we consider OpenRAIL licenses to be “open”. OpenRAIL enables reuse, distribution, commercialization, and adaptation as long as the artifact is not being applied for use-cases that have been restricted.
Our main aim is not to evangelize what is open and what is not but rather to focus on the intersection between open and responsible licensing."
FWIW, there's a lot of active discussion in this space, and it could be the case that e.g. communities settle on releasing code under OSI-approved licenses and models/artifacts under lowercase "open" but use-restricted licenses.
One possible suggestion for future iteration: I think it's a bit of a shame that everyone's been sharing ChatGPT outputs almost exclusively through screenshots. I threw together a (very quick and rudimentary) browser extension I've been using to save some of my more exciting transcripts in JSON files. I think something along these lines could be really great for sharing purposes, especially if people want to study these outputs more systematically (e.g. for research, or just a kind of crowd-sourced audit).
Here's the barebones extension, just as an example: https://github.com/nickmvincent/chatgpt-exploration/tree/mai...
There's also a longer discussion to be had about the best way to do this: ideally, we might save some of the formatting information too and/or more metadata (exact timestamps for responses, etc.), but I think the json with plaintext is adequate (at least for personal retrieval use).
This "paradox of reuse" is a really big deal, IMO (blog post on the topic: https://nmvg.mataroa.blog/blog/the-paradox-of-reuse-language...)
It's actually a separate concern from the spam / content bloat described in the linked post, but they complement each other in creating amplified harm for StackExchange-like platforms: some fraction of users may stop visiting the site (because ChatGPT answered without links) and some other fraction of users will submit spam to try and earn points.
It gives a generic answer that it's some proprietary combinations of "books, articles and websites". I'd guess Wikipedia is in there for sure (English and maybe other editions as well), something like "BookCorpus" (https://huggingface.co/datasets/bookcorpus), probably a large scrape of news articles up to 2021. And definitely a full scrape of pretty much the entire academic/scientific literature (just based on poking around). Overall, probably very similar to GPT-3 (which is also a bit mysterious still!)
The official post (https://openai.com/blog/chatgpt/) also describes that some pretty rich human feedback data was collected as well, for the reinforcement learning component. I think this probably the real secret sauce for why this feels so qualitatively different than a lot of the LLMs that came before.
The discussion linked in a comment above (https://meta.m.wikimedia.org/wiki/Wikimedia_Enterprise/Essay...) also provides a pretty nice FAQ to the negative responses.
Hill, B. M. and Shaw, A. (2021) ‘The Hidden Costs of Requiring Accounts: Quasi-Experimental Evidence From Peer Production’, Communication Research, 48(6), pp. 771–795. doi: 10.1177/0093650220910345.
It's always possible in the future that Rally, or other initiatives, will find other ways to incentivize contributions (more projects that align with specific interests, more "visualization" of how your data contributions might impact the world, etc.).
As to whether the cost-benefit accounting (producing new knowledge, informing design vs. scaring users off Firefox, creating new privacy harms) will come out in favor of the project remains to be seen, of course!
I'd like to highlight a few reasons why this could be exciting and, indeed "pro-user".
* The core value proposition in my view, though not spelled out on the linked page (I think because it's targeting a broader-than-HN audience), is that data collection can produce answers to empirical questions about "Big Tech" that would otherwise likely never be answered by anyone except in-house Big Tech data analysts. E.g., empirical questions about the impacts of search engine design choices, how newsfeeds influence behavior, etc. Of course, these studies will face issues with selection bias, but that's not insurmountable, and probably better than nothing. For instance, publicly shared data about Pixel tracking, even from a biased sample, is better than having no data on that topic, IMO!
* The collaboration with academics and journalists is important because it means the results of each study will be shared. It's in the incentive systems of both groups to publish results. Best case, individual users, people who build browser extensions, etc. can use findings about search engines and newsfeeds to change their behavior or build new tools. Policy discussions would also benefit from some quantitative grounding.
* The alternative ways to answer these questions: academics and journalists run scrapers, data-collection browser extensions, etc. on their own. Harder to do, especially post-Cambridge Analytica and more recently, NYU's ad observatory getting shut down (https://cyber.nyu.edu/2021/08/21/facebook-disables-ad-observ...). Or, the state forces tech co.'s to publish answers. Or, we just never know!
* A weaker point: This project is trying to set new standards about data management, e.g. providing a true "one click to opt out" option. I see this as an experiment, that other projects can follow if it works. This is a bit more of a stretch than the previous two points.
My bias: I'm an academic who wants to see the results of the various studies being conducted, so I'm very hopeful!
There's some assumptions underlying the theoretical result, but there's also a lot active experimentation in the real world, a few links: https://www.radicalxchange.org/concepts/quadratic-voting/, https://en.wikipedia.org/wiki/Quadratic_voting
Here's what I believe is the key paper on the topic https://www.aeaweb.org/articles?id=10.1257/pandp.20181002 (appendix: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2790624)
Lalley, Steven P., and E. Glen Weyl. 2018. "Quadratic Voting: How Mechanism Design Can Radicalize Democracy." AEA Papers and Proceedings, 108: 33-37.
A big upside of testing this out is the opportunity to keep better track of what services are Wikipedia-dependent. While search results often send people to Wikipedia (see e.g. https://diff.wikimedia.org/2021/09/23/searching-for-wikipedi...), voice assistants or Q&A systems might be (i.e. likely are) surfacing Wikipedia content in more subtle fashion. Same with large language models. Under status quo, it will be increasingly hard to track these relationships, so exploring alternatives is well worth it.
In general, a primary factor reviewers in computing conferences are asked to consider is the degree to which a submission make a "substantial contribution" to the community. What is or isn't a substantial contribution is subjective and entirely dependent on the prevailing ethical perspectives in various communities. Papers -- a key unit of academic progress (for better or worse) are entirely subject to concerns of ethics. Certainly, there are interesting argument around how much time and paper space should be spent on speculating about negative impacts, and people are having those conversations.
The fact that ethics is "trending" is because more researchers would like tackle ethical challenges explicitly, instead of falling back on the default of a given community. For instance, here is a paper that quantitatively (with reproducibility!) analyzes the values in ML papers. https://arxiv.org/abs/2106.15590 This is one way to have a very empirically-grounded discussion of the topic.
IMO, many researchers can and do debate topics of ethics in AI, and in doing so move the field forward (and increase likelihood that computing will have more positive impacts than negative ones).
I guess if that's what people believe, it may be tough to convince them otherwise... but IMO, it's really valuable to think about how the "winners" of history navigated tough moral challenges, and often did things that we can now identify as harmful.
In this case, it's not about making historical missionaries feel bad about themselves for "winning" -- this article (in my read, etc) is concretely about trying to avoid repeating these mistakes in modern tech, specifically NLP and translation. We can (hopefully) do better as a society than "might makes right"!