We had some google people come in at $CORPO_DAYJOB the other day to sell their cloud offerings. The engineer had the gall to say "people ask me, why did google miss the boat on LLMs? I say people, we BUILT the boat". While referencing "attention is all you need". Hilarious coping strategy guys, but because you had some good academics on your payroll in 2017, that doesn't mean you're delivering right now... we'll talk when you actually have something better than 90% of the other LLM offerings out there.

“X Built a boat == the builders remain at X until they die” Is a strange definition of building something imo.

When a company provides an R&D lab, we still attribute the results of that research to the company as well as to the researchers.

After all, if those individuals were not hired that position would still be filled with someone similar, but it's hard to argue that if the lab didn't bring those researchers together, they may have individually gotten the same result.

Honestly, it's not as easy as one may think to build a research lab.

A blog post on the internet is also copyrighted! Google has probably the largest collection of digitized books as well.

They have no reason not to. ML networks have been trained on copyrighted data since before 2012.

Ethics?

This response is copyrighted, all rights reserved.

But you can still read and gather information from it. Your computer still stores a copy of it when you reload the page. Theft? Infringement? I think not.

Is taking the character count of a book copyright infringement? Why is the math behind an LLM different?

I hate arguments like this. Intent, scale, and quantity matter.

If I gently toss a small piece of lead at you, I'm not trying to kill you. If I accelerate it using a rifle, I am.

No, taking the character count of a book is not copyright infringement. Neither is quoting the word "the". Or "the worst". But, "It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair." definitely is.

The fact that you have to draw a line and exercise some amount of judgement to determine which actions are prohibited does not mean the line doesn't exist.

Likewise, the social context and emergent consequences of a policy matter too.

If LLMs were a niche technology used by a couple of fringe weirdos, who cares, go nuts. But since LLMs are owned by large tech companies and are being used to generate incredible amounts of revenue, they are enacting a large-scale transfer of wealth to the rich whose value is largely derived from the labor of the creative people who authored the data the model is trained on without any consent on their part.

How you can't see that that is somehow different from taking the word count of a file is beyond me.

The response could contain information that you did not specifically refer to but is relevant and/or related. It doesn't even imply that the LLMs had access to the book's content.

Here the link with the answer of ChatGPT: https://chat.openai.com/share/7c46a3bd-efa8-49e9-8018-554e1e... and also a follow up question.

Which, most likely, implies that chatgpt was trained and retains actual book content rather than random people talking about it on the internet.

Which model is more likely to be in violation of some copyright?

On a completely unrelated point, why are people's tests of LLMs clearly designed to make me think that people are dumber? Are the LLMs suggesting these questions as part of their plot?