News for hackers. Stuff that matters.
Re:NY times is asking that all LLMs trained on Times data be... by grantseltzer on Saturday December 30, 2023 @01:05PM (#38817206)
Can someone with actual fundamental understanding of LLMs explain to me why they think it's perfectly legal to train models on copyrighted material? I don't know enough about this. Please don't answer by asking chatgpt.
Re:NY times is asking that all LLMs trained on Times data be... by Ukv on Saturday December 30, 2023 @01:18PM (#38817378)
Consider how commercial search engines are fine to show text snippets, thumbnails and site caches.

AI developers will most likely rely on a Fair Use defense. I think this has a reasonable chance of success since, while the use of a given copyrighted work may affect the market for that work (in this case NYT's article), it can be argued to be highly transformative usage. As in Campbell v. Acuff-Rose Music: "The more transformative the new work, the less will be the significance of other factors", defined as "whether the new work merely 'supersede[s] the objects' of the original creation [...] or instead adds something new".

There's also potential for an "implied license", as in Field v. Google Inc for rehosting a snapshot of a site, where "Google reasonably interpreted absence of meta-tags as permission to present 'Cached' links to the pages of Field's site". As far as I can tell in this case, NYT's robots.txt of the time was obeyed, which permitted automated processing of all but one specific article for some reason.

Re:NY times is asking that all LLMs trained on Times data be... by HumblyTossed on Saturday December 30, 2023 @01:33PM (#38817530)
> AI developers will most likely rely on a Fair Use defense.

Probably. The question for the courts to decide, then, is how much use is considered fair use.

Re:NY times is asking that all LLMs trained on Times data be... by mmh0000 on Saturday December 30, 2023 @04:06PM (#38819134)
I'm essentially a meat-based LLM and I'm trained almost exclusively on copyrighted material (most of which I pirated).
Re:NY times is asking that all LLMs trained on Times data be... by jibe on Saturday December 30, 2023 @01:13PM (#38817317)
Why do you think it is legal to train students on copyrighted material? Copyright is supposed to protect from unauthorized reproduction, not unauthorized learning. That the NY Times is able to show some verbatim reproduction, it is a real legal issue, but that should not be extended to training generally.
Re:NY times is asking that all LLMs trained on Times data be... by arduanika on Saturday December 30, 2023 @01:56PM (#38817801)
Students are humans. LLMs are not. Machine "learning" is a metaphor, not what's actually happening. Stop anthropomorphizing, and show some loyalty to your species.
Re:NY times is asking that all LLMs trained on Times data be... by gaganyaan on Saturday December 30, 2023 @05:10PM (#38819750)
Bizarre loyalty argument aside, why is it not learning? Can you quantify that statement?
Re:NY times is asking that all LLMs trained on Times data be... by mhss on Saturday December 30, 2023 @06:01PM (#38820144)
The loyalty argument does sound somewhat bizarre, but I think the overarching point is about whether technology use benefits humans in society or not. We should not implicitly treat LLMs owned by corporations with the same rights as humans. LLMs without some form of legislation is looking like it will benefit corporations that are salivating at profits and the prospect of reducing or eliminating the number of creative workers they need.
Re:NY times is asking that all LLMs trained on Times data be... by arduanika on Saturday December 30, 2023 @08:52PM (#38821068)
Why would I want to quantify it? The burden of proof is on the thief.

I have a gadget that will, with some probability, steal your life's savings. It operates through a process that is analogous to a human chewing. When engineering it, we just say for simplicity that the gadget "chews". Of course, that's only a metaphor -- machines can't chew.

But (and here's where your argument gets ridiculous), unless you can quantify the fact that my gadget can't chew, then I will steal your savings. Good luck.

Re:NY times is asking that all LLMs trained on Times data be... by gaganyaan on Sunday December 31, 2023 @12:53AM (#38822141)
Why can't machines chew? That's an even weirder analogy, it would be quite easy to make a machine that chews exactly like humans do.
Re:NY times is asking that all LLMs trained on Times data be... by rickydroll on Sunday December 31, 2023 @11:14AM (#38825043)
> I have a gadget that will, with some probability, steal your life's savings.

I can think of 2 instances of that machine already. the finance industry fees and an ex-wife.

Re:NY times is asking that all LLMs trained on Times data be... by pointlessone on Saturday December 30, 2023 @01:19PM (#38817390)
I think your question is incorrect. It’s very likely no-one thinks it’s perfectly legal. There probably are many people who think it’s not a big deal, though. Try coming up with a dataset that doesn’t have any copyrighted material in them. Like seriously try. You can’t use pretty much anything newer than a century old. Everything is copyrighted by default. Very few new things are explicitly in public domain or licensed in a way that would allow usage. Now imagine LLMs trained on early 20th century newspapers, books and letters. Do you think it would be good at generating code or hip copy for homepage of your next startup?
Re:NY times is asking that all LLMs trained on Times data be... by everforward on Saturday December 30, 2023 @01:39PM (#38817589)
> Now imagine LLMs trained on early 20th century newspapers, books and letters. Do you think it would be good at generating code or hip copy for homepage of your next startup?

Not sure about the rest of the world, but at least for US content I don't think any company would publish that LLM.

That's like 40 years before the civil rights movement, and right about the time of the Tulsa massacre.

It's right around when women got the right to vote.

Trying to get it to not say anything horrible under modern standards seems fraught with issues. I don't know if it would even understand something like "don't be racist", given the context it was trained on.

Re:NY times is asking that all LLMs trained on Times data be... by pointlessone on Saturday December 30, 2023 @03:46PM (#38818933)
Exactly. Copyright terms are so long that most material with expired copyright is not useful for modern uses of LLMs and looking for modern non-copyrighted materials is too hard to do quickly and its usefulness is unclear. So people who grew up with Internet and are used to making memes with copyrighted material are not exactly averse to do it on a bigger scale.
Re:NY times is asking that all LLMs trained on Times data be... by tmikaeld on Saturday December 30, 2023 @01:26PM (#38817451)
> Try coming up with a dataset that doesn’t have any copyrighted material in them.

Isn't this what Mistral AI did?

Re:NY times is asking that all LLMs trained on Times data be... by pointlessone on Saturday December 30, 2023 @01:50PM (#38817728)
Did they? That'd be interesting to take a look at. Do they publish contents of their dataset?
Re:NY times is asking that all LLMs trained on Times data be... by tmikaeld on Monday January 01, 2024 @03:38AM (#38830466)
The RAW Weights here: https://docs.mistral.ai/models/
Re:NY times is asking that all LLMs trained on Times data be... by Manuel_D on Saturday December 30, 2023 @01:55PM (#38817780)
Fair use, probably. How many news pieces have you read that amount to, "The New York Times reports..." followed by a summary of the Times' article? It's not illegal to use copyrighted works at a source, as inspiration, or to guide style.
Re:NY times is asking that all LLMs trained on Times data be... by londons_explore on Saturday December 30, 2023 @01:17PM (#38817364)
I think the main arguments are:

1. Training an LLM is akin to human learning. It is legal to read a textbook about music to learn music, and later to write a book about music which likely includes some of the concepts you earlier learned.

2. Neither the LLM nor the output text contain sufficient elements of the copyrighted work to qualify for copyright protection. Just like if you turned old library books into compost and sold the compost, you wouldn't expect to pay authors of those books a royalty for the compost sales.

Re:NY times is asking that all LLMs trained on Times data be... by madeofpalk on Saturday December 30, 2023 @02:09PM (#38817974)
> Training an LLM is akin to human learning. It is legal to read a textbook about music to learn music

If you learn a little too hard though, and reproduce the original textbook in it's entirety, you'll get in trouble.

My guess is that courts will determine that the training itself will not be found illegal, but either the AI companies, or the users, will be found liable for reproducing copywrighted work in output, and no one will want to hold liability for that.

Re:NY times is asking that all LLMs trained on Times data be... by HDThoreaun on Saturday December 30, 2023 @01:41PM (#38817610)
I feel like there’s no way 1 will fly. Very soon ai and humans will explicitly have to follow different laws because they operate very differently.
Re:NY times is asking that all LLMs trained on Times data be... by formercoder on Saturday December 30, 2023 @01:09PM (#38817261)
I, a human, can read a copyrighted work and then write a new work and own the copyright on that new work as long as it is not substantially the same.
Re:NY times is asking that all LLMs trained on Times data be... by foogazi on Saturday December 30, 2023 @01:14PM (#38817331)
What if you produce a substantially similar work ?

Who owns the copyright then ?

Re:NY times is asking that all LLMs trained on Times data be... by Vvector on Saturday December 30, 2023 @01:28PM (#38817471)
If the work goes beyond fair use, it is a copyright violation. It doesn't matter if it was created by a person or an AI.

Technology that makes copyright violations easier/quicker have typically been found legal if "the technology in question had significant non-infringing uses".

Re:NY times is asking that all LLMs trained on Times data be... by karmakaze on Saturday December 30, 2023 @02:06PM (#38817924)
This makes sense. It was allowed for the content to be read and used in certain ways (e.g. search engines or as references) without substantial reproduction. The NYT would then have to flag specific generated content as infringing a specific work which could then be judged as fair use or not on a case-by-case basis. If a particular site/company was repeatedly and/or primarily using substantial content then perhaps it could be 'delisted' as search engines do for links to pirated copies of works.
Re:NY times is asking that all LLMs trained on Times data be... by michaelmrose on Saturday December 30, 2023 @01:26PM (#38817453)
It really hinges on substantially similar. If I copy Harry Potter and change every instance of Harry Potter to Michael Rose surely it's infringing. If I write a coming of age story set in a magical land I'm probably OK. Which do you think LLM produce?
Re:NY times is asking that all LLMs trained on Times data be... by foogazi on Saturday December 30, 2023 @05:56PM (#38820103)
Whatever you ask and the model will be judged by its accuracy.

If you ask for Harry Potter and it gives you Bart Simpson it’s useless.

Re:NY times is asking that all LLMs trained on Times data be... by michaelmrose on Saturday December 30, 2023 @06:39PM (#38820401)
It's likely not possible of literally giving you Harry Potter. If you specify it narrowly enough that it qualifies as fan fic its probably exactly what you were going for. After all your word processor is capable of producing infringing works but is not itself an infringing work.
Re:NY times is asking that all LLMs trained on Times data be... by endisneigh on Saturday December 30, 2023 @01:08PM (#38817243)
It’s currently not explicitly legal or not. There are lawsuits in progress to determine that very answer.
Re:NY times is asking that all LLMs trained on Times data be... by MPSimmons on Saturday December 30, 2023 @01:12PM (#38817304)
Read about the 'fair use' doctrine and put yourself in the shoes of someone who is training a model, and see if you can argue, from their perspective, why it should be allowed.
Re:NY times is asking that all LLMs trained on Times data be... by michaelmrose on Saturday December 30, 2023 @01:24PM (#38817429)
Surely. Remember when the VCR came out and some parties absolutely freaked out and Jack Valenti said

"I say to you that the VCR is to the American film producer and the American public as the Boston strangler is to the woman home alone."

Then we invented from whole cloth reasons why they were perfectly OK because there was a ton of money to be made and everyone would actually be better off if the VCR was a thing and everyone knew it because it ended up argued after millions of VCRs were already in households.

Re:NY times is asking that all LLMs trained on Times data be... by Vvector on Saturday December 30, 2023 @01:30PM (#38817495)
I was thinking of the VCR as well. SCOTUS ruled that "the technology in question had significant non-infringing uses" making VCRs legal.
Re:NY times is asking that all LLMs trained on Times data be... by FpUser on Saturday December 30, 2023 @01:11PM (#38817283)
We all "train" ourselves on copyrighted materials and later use / or not gained knowledge for our own benefit be it financial or pleasure.

They're just on a hunt for some extra money.

Re:NY times is asking that all LLMs trained on Times data be... by arduanika on Saturday December 30, 2023 @02:09PM (#38817977)
Humans aren't computers. Come on, people.