This is an argument in bad faith but at this point I have zero trust in corporations and feel like you can generally count on them to do shitty things if they can benefit from it so I can be easily swayed by little proof at this point.

What's the argument? What's been done by anyone that's shitty? I don't even understand the point of this post. As far as I know, the current wave of text-based AIs is trained on all text accessible on the internet. Would it be a scandal to learn that ChatGPT is trained on wikipedia? Reddit? What is even the argument here, good faith or otherwise?

The argument is these companies are using our ideas created by us humans in this thing called the interenet for free and without attribution and it's problematic.

I'm not necessarily arguing against you, but "problematic" is too generic a term to be useful. Genocide is "problematic". Having to run to the bathroom every 5 minutes to blow my runny nose is "problematic". What do you actually mean?

You can't own ideas, they got their own life-cycle.

Right, but I do think you can "own" (by which I mean our societally-mediated legal definition of ownership in the anglosphere) specific sequences of text or at least the right to copy them?

Responding to sibling comment: We need some clarification here: are we speaking about just ideas in the abstract sense, or ideas that have been fleshed out i.e "materialized"

If the latter, there are many laws that say you can own an idea, provided it exists somewhere.

From an open source point of view it would be better if scraping proprietary LLMs would be allowed. Small LMs need this infusion of data to develop.

But the big news is that it works, just a bit of data can have a large impact on the open source LLMs. OpenAI can't have a moat in their proprietary RLHF dataset. Public models leak, they can be distilled.

What shitty things are you talking about?

But remember many years back when it was news that Bing used Google search results to improve its results.

> Google catches Bing copying [search results], Microsoft says “so what?”

https://arstechnica.com/information-technology/2011/02/googl...

It's not quite the same thing, because Bing was getting the data from a browser toolbar and watching the search terms used and where the user went afterwards.

A closer equivalent would be if someone had made a ShareSERP site and people posted their favorite search terms and the results Google gave and Bing crawled that and incorporated the search terms to links connections into their search graph.

The actual actions had maybe gone too far (personally I thought it was more funny than "copying"), the hypothetical would be pretty much what you'd expect to happen. Even google would probably crawl ShareSERP and inadvertently reinforce their own results (the same way OpenAI presumably gets more than a bit of their own results back at them in any new crawls of reddit, hn, etc even if they avoid sites like ShareGPT deliberately).

OpenAI Terms of service forbid training competitor models via their ML outputs (LoRa alpaca laundering is probably not allowed for commercial use).

Google has no contract with OpenAI though. They used a third party site to scrape conversations. If the outputs themselves are not copyrighted, and they never agreed to the terms of service, it should be fine, right? Albeit unethical and embarrassing.

> Albeit unethical and embarrassing.

I really don’t understand this angle. In fact, I am fairly positive that the training set for GPT-4 contains many thousands of conversations with AI agents not developed by OpenAI.

Do AI companies need to manually sift through the corpus and scrub webpages that contain competitor LLM output?

(“Yes” is an acceptable answer to this, but then it applies to OpenAI’s currently existing models just as much as to Bard)

How did you come about being "fairly positive" that GPT-4 is trained on other AI conversations?

Many AI conversations have been floating around internet forums since the original GPT was released. As OpenAI hasn't shared anything about its training set, to err on the side of caution I would assume that they didn't filter these conversations out. If they aren't even marked as such, it may not even be possible to do. I think it would be very hard to prove that no AI conversations are included in the training set, even if it wasn't secret.

Hardly unethical, considering OpenAI is doing exactly this.

You can quibble about the ethics of web scraping for ML in general but I think you're conflating issues.

OpenAI and Google both scour the web for human-generated content. What Google cares about here is the learnings from OpenAI's proprietary RLHF dataset, for which they had to contract a large sum of human labelers. Finding a roundabout way to extract the value of a direct competitor's purpose-built, costly data feels meaningfully different from scraping the web in general as an input to a transformative use.

> OpenAI and Google both scour the web for human-generated content

OpenAI and Google both scour the web for content, period. That content could be human generated or AI generated or a mix of the two. Neither company is respecting copyright or terms of service of every individual bit of data collected. Neither company cares how much effort was put into creating the data, whether humans were paid to do it, or whatever else. So there really isn't that much difference between the two.

And herein is the main problem of AI. Its creators consume knowledge from the commons, and give nothing free and unencumbered back.

It's like the guy who never brings anything to the potluck, but after everyone finishes eating, he boxes up the leftovers, and starts selling them out of a food cart.

> after everyone finishes eating, he boxes up the leftovers, and starts selling them out of a food cart.

That particular example doesn't seem all that great, as it gives the impression of not letting food go to waste.

Though sure, the guy could just give the food away, but shrug.

So what? Is OpenAI RLHF dataset more valuable than millions of books and paintings OpenAI used for free without stopping a second? Why is that? Because one big tech corp paid money for that dataset?

If there's a party which has intentionally conflated scraping web content in general with scraping it to build a direct competitor to the original sources, that party is Google.

Yes, this latest instance with OpenAI outputs is shady, but I think it's in the same spirit as scraping news organizations for content which journalists were paid to write, and then showing portions of it directly in response to queries so people don't go directly to the news organization's pages, and it's in the same spirit as showing answers to query-questions that are excerpts from scraped pages which another organization paid to produce.

I see no difference. Any web scraping is a means to deflect revenue-generating traffic to yourself, and away from other websites. Fewer people will go to Stack Overflow because of Codex and Copilot. The point that the content was paid for vs volunteered becomes moot once it's posted publicly online for free, on ShareGPT.

> labelers. Finding a roundabout way to extract the value of a direct competitor's purpose-built, costly data feels meaningfully different from scraping the web in general as an input to a transformative use

There we go again, its, one law for the unwashed plebs and the other for us.

Why do you think that I, after spending my time and effort to write my blog, own my content to a lesser extent that OpenAI does their? Such hypocracy.

Two wrongs don’t make a right.

forgive me if i have limited sympathy when a burglars house gets robbed

This

It's even less worthy of sympathy - like a counterfeit piece of art being counterfeited. And there isn't even an original, just like a made up counterfeit.

That's not how rights work, though. I'm sure you don't want your rights to be conditional on the sympathy of others.

good thing I'm just some guy expressing an opinion, and not a judge, then

It’s still debatable if training a computer neutral network on public data is 'wrong' when we very much accept it as a right for biological neural networks.

No more unethical or embarrassing than scraping the web for millions of copyrighted works and selling access to unauthorized derivative works.

The recent HiQ vs LinkedIn case would seem to make this ToS unenforceable, unless Google actually created a user account on ShareGPT and affirmatively accepted the terms. "Acceptance by default" does not count, and I can easily browse ShareGPT without affirmatively accepting any ToS, without which web scraping is totally legal.

So, to verify, are you claiming that if someone added a similar clause to their source code and then GitHub went ahead and trained Copilot against it, that would be an issue?

You relinquish all licensing rights when you upload your code to GitHub. Microsoft can do whatever they want with it. That's in their ToS, which you have to agree to when you make an account. Normally, only affirmatively accepted ToS are enforceable, so just putting a clause into your license doesn't work (unless it's a copyright, which doesn't require consent).

> You relinquish all licensing rights when you upload your code to GitHub

What now? Seriously?

I found this. Section D4.

"We need the legal right to do things like host Your Content, publish it, and share it. You grant us and our legal successors the right to store, archive, parse, and display Your Content, and make incidental copies, as necessary to provide the Service, including improving the Service over time. This license includes the right to do things like copy it to our database and make backups; show it to you and other users; parse it into a search index or otherwise analyze it on our servers; share it with other users; and perform it, in case Your Content is something like music or video."

"as necessary to provide the Service" seems critical.

Also, section D3 of the GitHub Terms of Service says:

> You retain ownership of and responsibility for Your Content.

and section D4 says:

> This license does not grant GitHub the right to sell Your Content. It also does not grant GitHub the right to otherwise distribute or use Your Content outside of our provision of the Service, except that as part of the right to archive Your Content, GitHub may permit our partners to store and archive Your Content in public repositories in connection with the GitHub Arctic Code Vault and GitHub Archive Program.

There is nothing in the terms that requires the GitHub user to relinquish all licensing rights.

https://docs.github.com/en/site-policy/github-terms/github-t...

The clauses always have a trap door: "[outside of] our provision of the Service" means they can do anything as long as it's a service they provide.

Under definitions: The “Service” refers to the applications, software, products, and services provided by GitHub, including any Beta Previews.

I think there's misunderstanding over what the word "relinquish" means.

The terms make clear that uploading code to GitHub gives GitHub the right to "store, archive, parse, and display Your Content, and make incidental copies, as necessary to provide the Service, including improving the Service over time" while the code is hosted on GitHub.

However, that's not the same thing as relinquishing (giving up) licensing rights to GitHub. The uploader still retains those rights, and there is nothing in the terms that says otherwise.

The question turns on whether you consider copilot part of the "GitHub service."

GitHub would argue that it is, and they'd likely argue that charging for access to copilot is akin to charging for access to private repositories.

Others would say that copilot is somehow separate from the services Github provides, so using their code for CoPilot wouldn't be covered by the ToS.

It is certainly a service that's being provided. If not by GitHub, then by whom?

I'll repeat the definition of service: The “Service” refers to the applications, software, products, and services provided by GitHub, including any Beta Previews.

So do you believe if you hosted a closed source project on GitHub, and GitHub decided they want to integrate this into their service they would simply be allowed to take the code?

Fortunately HN commenters are not judges. And I would wager any bet that MS lawyers would not try to argue based on their ToS either, that would be a recipe for loosing any court case.

I just mean that it doesn't really matter what your license says as long as GitHub can come up with a business justification for using it in some way. Certainly, other users still legally have to obey your copyright.

"Improving the service over time" can do a lot of heavy lifting, definitely including training Copilot.

So, to verify, are you claiming it would not be allowed for you to upload my otherwise-open-source code (code I do not myself host at GitHub, but which was reasonably popular / important code) to GitHub?

Yep. It's in their ToS:

If you're posting anything you did not create yourself or do not own the rights to, you agree that you are responsible for any Content you post; that you will only submit Content that you have the right to post; and that you will fully comply with any third party licenses relating to Content you post.

I suppose this means if I upload your stuff to GitHub, and you sue GitHub, then GitHub would be able to somehow deflect liability onto me.

That doesn't make sense. For example, GPLv3 allows anyone to redistribute the software's source code if the license is intact:

> You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program.

https://www.gnu.org/licenses/gpl-3.0.en.html

If GitHub uses the source code in a way that violates the license, there is no provision in the GitHub terms of service that would allow GitHub to deflect legal liability to the GitHub user who uploaded the program. The uploader satisifed the requirements of GPLv3, and GitHub would be the only party in violation.

Uploading is granting GitHub a license separate from the gpl license.

If you can't actually grant that separate license, you're misrepresenting your ownership and license to that code

I'd like to see that theory tested in court. Section D3 of the terms says:

> If you upload Content that already comes with a license granting GitHub the permissions we need to run our Service, no additional license is required.

https://docs.github.com/en/site-policy/github-terms/github-t...

and section D4 does not mention any permissions that GPLv3 does not already cover. GitHub automatically recognizes when a repo is GPLv3-licensed, so it cannot claim ignorance of what GPLv3 is.

I love it how they don't want others to use their model output but they have no qualms about training their model on the copyrighted works of others? Isn't this a stunning level of hypocrisy?

breaking terms of service is not punishable in any way. Facebook tried and lost in court

Correction – breaking terms of service that you have not explicitly agreed to is not punishable in any way. A site cannot enforce a "by using this site you agree to..." clause deep inside some license page that visitors are generally unaware of. If you violate an agreement that you willingly chose to enter, however, you will likely be found liable for it.

Where exactly does it do that? I looked a bit and could t find it, but likely I was just wrong

Are the TOS even enforceable is AI content can't be copyrighted?

This is really hilarious. Authors and artists never gave permission to use their work to train AI models either...

Not legally the same situation, but ethically close enough.

> The output from ChatGPT is not copyrightable by OpenAI.

I think the argument here is over the OpenAI Terms of Service, not copyright.

And what about the terms of service of my blog or code repository? Does OpenAI respect that?

> And what about the terms of service of my blog or code repository? Does OpenAI respect that?

Seems to me that’s an issue between you and OpenAI. (Does your blog or code repository actually have published restrictive terms of service? Did it when OpenAI accessed it? Did OpenAI even access it?)

You think OpenAI is going to care unless you have a team of expensive lawyers to back you up?

Microsoft is out there laundering GPL code with Copilot. These companies live firmly in the don't give a fuck region of capitalism. Copyright law for thee, not for me.

Since it was through ShareGPT, is the argument like "what color are your bits" but for ToS?

Maybe if they had put in their terms of service "you can only share this on sites with their own ToS that allow sharing but disallow using the content for training models, and also replicate this requirement", I don't see how you could have any sort of viral ToS like that.

Seems more like it's just a bad idea to rely heavily on another LLM's output for training.

See HiQ vs LinkedIn. ToS has to be affirmatively accepted. I doubt that happened in this case.

>Even if they did – so what?

Amplification of biases, propagation of errors, echolalia and over-optimization, lack of diverse data, overfitting

I mean maybe. There also might be something to this. OpenAI has been very opaque about training techniques.

That's just the base concern with every single model regardless of where they sourced their data from. Garbage in, garbage out.

Sure. Does that fact mean we're prohibited from expressing concerns about data quality? ShareGPT isn't representative of authentic, quality writing.

Right, but training an LLM on the output of another LLM can certainly exacerbate these issues

Maybe, but we are fast approaching the point (or more likely have crossed it already) where distinguishing between human and AI generated data isn't really possible. If Google indexes a blog, how does it know whether it was written with AI assistance and therefore should not be used for training? Heck, how does OpenAI itself prevent such a feedback loop from its own output (or that or other LLMs)?

Your argument would have a lot more force if we were past that point rather than fast approaching that point. Concerns about training data errors being compounded are much more important when you're talking about the bleeding edge.

And your question about how OpenAI prevents their training data from being corrupted is one we should be asking as well!

> If Google indexes a blog, how does it know whether it was written with AI assistance and therefore should not be used for training

Yes, this is an existential problem for Google and training future LLMs.

See also, https://www.theverge.com/23642073/best-printer-2023-brother-... and https://searchengineland.com/verge-best-printer-2023-394709

Or Google can just materialize the expected page into existence at search time.

... it's uncanny how it always finds what you thought you were looking for!

> Heck, how does OpenAI itself prevent such a feedback loop from its own output (or that or other LLMs)?

Seems trivial. Only use old data for the bulk? Feed some new data carefully curated?

Future job: token selector / archiving

<meta name="generator" content="human brain">

I'm only half joking.... I think we likely will end up with flags for human generated/curated content (and it will have to be that way round, as I can't imagine spammers bothering to put flags on AI-generated stuff), and we probably already should have an equivalent of robots.txt protocol that allows users to specify which parts of their website they would and wouldn't like used in the training of LLMs.

Reminds me of the old “evil bit” RFC[1]

[1] https://www.ietf.org/rfc/rfc3514.txt

If content with a "human-generated" flag is rated more highly in some way -- e.g. search results -- then of course spammers will automatically add that flag to their AI-generated garbage. How do you propose to prevent them?

I assume, like the actual meta generator tags, it wouldn't actually be a massive boon for regular search results

And if it's not: why bother.

I think something like this will definitely happen, and your suggestion is the cleanest implementation idea I've seen for it. I imagine there will be a service provided by Google and OpenAI where they verify your identity as a human and then grant you a token to put into your meta tags (wait a second... this sounds like sama's worldcoin idea...).

It will need to be based somewhat on the honor system (just because someone's proved they're a human doesn't mean they won't put their attestation on auto-generated text), but it definitely sounds better than nothing.

They'll still need to incentivize it somehow, though. Why do I as a human want to add that meta tag? If the answer is "better search ranking" then it renders the whole scheme mostly pointless because obviously spammers will want to acquire the attestation and attach it to their auto-generated content.

Pre-AI data is going to become like pre-nuclear steel.

Not to mention it's embarrassing. Google playing second banana to OpenAI.

That assumes that training on the output of another language model somehow gives you the ability to improve your model and to catch up somehow

It does. In general this is known as teacher-student training or knowledge distillation. It works better if you have access to the activations of the model but you can work with just outputs as well.

Does it work even if you don't control the inputs?

Well, it does, that's how we got Alpaca from LLaMA.

I think Amazon was first in the (free) banana business

you joke, but first producy they changed on whole foods were the bananas.

before: organic (south america) and regular (central ou SEA) for 69, 59.

then: both chikita's brand with regular and organic stickers (clearly the same produce, always from SEA) for 49 and 39 cents.

thats was days after the announcement

Did you inadvertently reverse to regular/organic order, or was organic cheaper after?

Google’s been second banana to openai for a few years now, right?

The funny part is that deepmind's tech (and some of Google Brain's research) seems to be as good as openAI's or better, but Google's unwillingness/inability to productionize these systems is keeping them back. It seems like the issue is only with the management, and I'll be looking forward to reading about Google's version of Fumbling the Future[0].

[0] https://www.amazon.com/Fumbling-Future-Invented-Personal-Com...

Many of Google's products were second, or later, to market. Google does not care.

Isn't it generally very hard to be first to market? And even if you are it's more likely that someone coming in later will take your lunch.

Apple wasn't the first one to try to make a successful smartphone, but they had resources, know-how, and tried at a better time with fewer unknowns around.

Or were just willing to adapt when others didn’t. Blackberry mocked the touchscreen for years until finally coming around and their initial implementation was awful.

Blackberry Torch??

That's okay if they can leapfrog.

Google Maps wasn't the first map product. We all used mapquest way before. But Google Maps was technologically advanced. Ajax made maps usable for the first time.

Gmail wasn't the first webmail. Hotmail had millions of customers already. But Google gave people unlimited space to store old email, whereas email in the old days filled up your inboxes and needed to be deleted.

Question is if they can and will leapfrog.

Google Plus was a sign of desperation and utterly failed. The dozens and dozens of different Messengers (sorry I don't even know what the latest one they're pushing is, RCS?) all failed.

As an organization we will see in the coming months and years if Google can still overtake others when coming in from behind or not.

>But Google gave people unlimited space to store old email

This isn't correct. Gmail launched with 1GB per user, which was way higher than other services, and they did keep doubling the storage space year-after-year, but it was never unlimited until Google Apps offered unlimited storage for businesses and schools.

That is true. You're technically correct.

In 2004 1GB of email was effectively unlimited though. Keep in mind hotmail offered 2MB and only increased that 125x when Gmail launched.

https://www.cnet.com/tech/services-and-software/hotmail-to-o...

Ascribing actual human emotion to a giant corporation like a google is probably not a good idea. Their motivations aren't going be heavily dictated by feelings of shame at being late out of the gate.

you talk like chatgpt was some bastion of curated perfectly correct content. get a grip. web scraping is web scraping.

Regarding point 2, I think there's nothing "wrong" with it, mainly it's funny that they don't know how to do it themselves. Provides additional evidence that Google is outgunned in this fight.

Yup

The idea of doing this is embarrassing enough for Google.

Google index the whole web, some of the documents are due to be generated by ChatGPT, there is no way around it.

Seems to me like it makes Google look kind of pathetic. That's worse than any legal issue here. (Caveat: assuming I understand the situation correctly)

if ChatGPT trained using Bard data, this site would be LIT UP because of OpenAI's association with Microsoft.

but it's google so no big deal right?

> Google denies doing it

Read their statement carefully and it's actually not a denial of the allegation.

> But Google is firmly and clearly denying the data was used: “Bard is not trained on any data from ShareGPT or ChatGPT,” spokesperson Chris Pappas tells The Verge

* Allegation: Google used ShareGPT to train Bard.

* Rebuttal: The current production version of Bard is not trained on ShareGPT data

Both things can be true:

* Google did use ShareGPT to train Bard

* Bard is not currently trained on any data from ShareGPT or ChatGPT.

It depends on what the meaning of is is ;)

Intent matters I guess.

Did they accidentally train on that public piece of info they scraped anyway because they are scraping the whole web?

Or did they intentionally scrape chatgpt output to see if that would help?

They could have trained, then modified code, repeat, to better enhance training in the current version.

Then after, train on raw data.

Trained would mean the current model wasn't trained at all from ShareGPT data, not that was trained on it previously, and isn't being trained anymore.

This association makes no sense.

Ok, I've added that information to the title—thanks. There's also https://www.theverge.com/2023/3/29/23662621/google-bard-chat....

Unfortunately the original report (https://www.theinformation.com/articles/alphabets-google-and...) is hardwalled.

The Verge doesn’t have an Information sub?

Whether or not training one giant LLM with socially enormous stakes on the output of another commercially-controlled LLM is an interesting question.

My stronger opinion is that the people who can do this stuff via having a crawled corpus of the Internet need to keep in mind that it's all our "user-generated content" that they've freely appropriated to build their models, and so whatever the technical copyright rules are (or become): you don't ethically own something that's closely imitating stuff we all wrote over the years.