News for hackers. Stuff that matters.
Re:Llama and ChatGPT Are Not Open-Source by bick_nyers on Thursday July 27, 2023 @06:13PM (#36900844)
I personally do not want the companies to release training data (at least for a while) because then it gives people leverage to neuter it.

I don't want a sanitized LLM, and I don't have $60M lying around to train my own.

Copyrighted material, sexual content, political opinions, throw it all in and release it please!

Yes, reducing bias in the models is a noble goal, but introducing new bias and blindspots to do it is a no-no.

Maybe I just got added to a list somewhere for having this opinion.

Re:Llama and ChatGPT Are Not Open-Source by threeseed on Thursday July 27, 2023 @06:23PM (#36900945)
> reducing bias in the models is a noble goal

It's also a necessary goal in order for these models to be more broadly adopted.

We've seen too many examples of bias in the training data set manifesting in ways that actively discriminate against people. Which is unethical and in many places illegal.

And having copyrighted material and sexual content in your model will simply open you up to lawsuits as is happening right now between authors and OpenAI. Not sure that is a position most startups want to be in.

Re:Llama and ChatGPT Are Not Open-Source by version_five on Thursday July 27, 2023 @06:38PM (#36901070)
Nah, these are foundation models. Companies want to be able to put in guard rails that are applicable to their application, not start with a model lobotomized according to US tech company values. The censorship is about telling people how to think, like it always is, not for the good of the people using the models.
Re:Llama and ChatGPT Are Not Open-Source by bick_nyers on Thursday July 27, 2023 @06:46PM (#36901125)
Yup. Advance the foundation models as far as possible to create a representation of the internet/human experience and place safeguards on top.

Don't want your LLM to be used to create erotic fanfiction? Instead of gutting the LLM, just put a detector on the query and answers feeding into the LLM. Thankfully, with a non-neutered model you have access to a tool than can be used to perform such detection...

Re:Llama and ChatGPT Are Not Open-Source by pkdpic on Thursday July 27, 2023 @07:27PM (#36901433)
Or tell another instance of the model to act as a constitutional-driven censor and let them duke it out.
Re:Llama and ChatGPT Are Not Open-Source by bick_nyers on Thursday July 27, 2023 @07:32PM (#36901476)
Now that's a fanfic I would read!
Re:Llama and ChatGPT Are Not Open-Source by threeseed on Thursday July 27, 2023 @06:48PM (#36901154)
> US tech company values

Which do broadly align with the values across most of the world.

But if you want to build something that has different values then go ahead.

But you can't expect companies to be complicit in doing something which is unethical or illegal.

Re:Llama and ChatGPT Are Not Open-Source by goatlover on Thursday July 27, 2023 @07:20PM (#36901378)
> Which do broadly align with the values across most of the world.

Satire?

Re:Llama and ChatGPT Are Not Open-Source by version_five on Thursday July 27, 2023 @07:22PM (#36901400)
Colonialism
Re:Llama and ChatGPT Are Not Open-Source by bick_nyers on Thursday July 27, 2023 @06:37PM (#36901064)
College students don't want to be criminally charged with theft for pirating an $800 college textbook either. I'm not giving business advice, I'm giving humanity and knowledge propagation advice.

As I mentioned, reducing bias is good so long as it doesn't introduce more bias elsewhere. The ultimate goal of course being a 100% bias free model.

Re:Llama and ChatGPT Are Not Open-Source by version_five on Thursday July 27, 2023 @06:18PM (#36900904)
100% agree. People that want training data released mostly just want to find something to attack, it has nothing to do with transparency.
Re:Llama and ChatGPT Are Not Open-Source by kristopolous on Thursday July 27, 2023 @06:47PM (#36901143)
We really need to somehow separate a bias towards accuracy as distinct from some bias towards say, a sports team.

Using everything would be like taking a bunch of students final exams and then claiming the most common answers are the correct ones.

This isn't how expertise and accuracy works. Most things worth doing are not only genuinely hard and complicated but something that only a minority subset of accomplished people can do consistently well in.

Listening to everybody and incorporating their thoughts is only going to lead to wrong answers.

Being selective is the key here unless you genuinely want say, answers about to space to involve aliens and UFOs - because way more people believe in that then there are qualified PhD astrophysicists in the world.

Similarly, way more people believe in vaccine conspiracy theories then there are people with significant viral epidemiology backgrounds.

This pattern is true in every field.

Re:Llama and ChatGPT Are Not Open-Source by bick_nyers on Thursday July 27, 2023 @07:00PM (#36901231)
Why is it a binary choice? "Most students would answer X, due to this common misconception about Y".

As we have seen from history, there is not often an absolute truth to questions, only clusters of truths. We want our LLM to be able to perform reasoning, mathematics, and science, but expecting absolute truths in anything outside of those fields is a bit much. Wikipedia often takes a good approach here and strikes this balance well. You can represent the "crazies" and show their reasoning, and then draw attention to how it is commonly refuted. You don't get this if you just omit the "crazies" to begin with.

Edit: What I'm saying is that you can include/represent these adjacent clusters of answers instead of asserting one cluster to be true and allow the correct answer to be represented organically. And guess what? Some people will still disagree for whatever reason they have to disagree. We should be taking all this AI alignment money and putting it into education imo.

Re:Llama and ChatGPT Are Not Open-Source by kristopolous on Thursday July 27, 2023 @08:06PM (#36901720)
That's not how bullshit works.

An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real.

If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever.

The people working with these fictions can always just add more fiction to fill in whatever hole. They aren't bound by reality or credulity.

Actually understanding global climate systems takes an incredible amount of study. You just can't blurt out a few paragraphs of words and get people to stop believing in Jewish weather machine conspiracies or whatever they're thinking.

We would have been done with that by now if this were the case.

Dislodging bullshit is hard. Humanity only started figuring out how a few hundred years ago and we're still fucking it up - such as in the replication crisis

Re:Llama and ChatGPT Are Not Open-Source by bick_nyers on Thursday July 27, 2023 @08:25PM (#36901845)
Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already.

If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason.

Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.

Re:Llama and ChatGPT Are Not Open-Source by eropple on Thursday July 27, 2023 @08:28PM (#36901870)
> Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already.

Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet.

The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."

Re:Llama and ChatGPT Are Not Open-Source by bick_nyers on Thursday July 27, 2023 @08:37PM (#36901939)
Just because it is a probability doesn't mean that it's strictly the probability derived from frequency in your training corpus. If you just used that as your model, it would not be nearly as good. Things like reasoning, a world model, and other heuristics can be used to generate said probabilities internally.

All that being said, who is to say that humans aren't just looking at probabilities when they think?

Re:Llama and ChatGPT Are Not Open-Source by kristopolous on Thursday July 27, 2023 @08:34PM (#36901919)
Exactly. Especially when there's some hustle behind lying to people. Phony medical advice is the classic one here. People do it to sell their sugar pills or do some Peter Popoff style healing touch hustle.

Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated.

The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material.

Re:Llama and ChatGPT Are Not Open-Source by ta93754829 on Thursday July 27, 2023 @08:04PM (#36901707)
When you dig into it, there are very few truths. Who "won" the last US election? Where did covid come from?

Humans will argue the right answer until their last days. It's frustrating how on-the-fence chatgpt can be. It's pretty interesting too, because in a professional environment one of the most important things you need to do is have an opinion and take a position otherwise you can't execute.

Re:Llama and ChatGPT Are Not Open-Source by kristopolous on Thursday July 27, 2023 @08:29PM (#36901882)
That's not true in the slightest. Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go.

To English learners, that might be very difficult. I'm learning a new language now and I feel the pain. The textbook and instructor, in this case, is way more correct than the collective opinion of my fellow students.

The vast majority of things actually follow this pattern instead. Knowing what a programming function would output given a set of inputs isn't some gray area world of opinion.

When things are unknown, such as say Covid origin, again there's people way beyond my pay-grade doing investigative work here. Instagram comment threads and tiktok opinions are actually not as valuable as peer reviewed scholarship on the topic EVEN IF there's more instagram comments saying one thing than papers in the Lancet saying another. The Instagram commenters don't become correct by volume here.

It's not some giant equal playing field.

Re:Llama and ChatGPT Are Not Open-Source by reverius42 on Thursday July 27, 2023 @10:23PM (#36902520)
> Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go.

As a linguistic descriptivist, hahahahahahahaha.

Re:Llama and ChatGPT Are Not Open-Source by trealira on Friday July 28, 2023 @01:16AM (#36903311)
Are you saying that the way people naturally speak follows no rules whatsoever?

Linguistic descriptivism just means that, rather than hold up an ideal of a language and prescribe variants as wrong or right, linguists should simply describe the language based on its use. That doesn't mean that the language (or variants of it) has no grammatical rules. It just means that, rather than holding up some prestige variety as "the language," and unprestigious ones as "uneducated errors" that shouldn't be studied, that you study all of them and determine how they work and how they're developing.

No matter where you go, people speak languages with a limited set of phones, which are mapped onto morphemes, which combine by particular rules to form words, which themselves form larger groups, like phrases and sentences (often the line between these things isn't clear cut). But languages all have rules of their own.

To imply that descriptivism means languages have no rules would be like saying that physics has no laws, because a physicist makes empirical observations instead of just deciding whatever the laws of physics ought to be.

Re:Llama and ChatGPT Are Not Open-Source by webmaven on Friday July 28, 2023 @06:52PM (#36914453)
I think the claim isn't that "language has no rules" but rather that the "rules" are far more context dependent and subject to change over time than would ordinarily be implied by the use of the word "rules".

Even at the lowest level of phonemes to morphemes, accents introduce a huge amount of variety, and it doesn't really get more orderly as you increase the level of abstraction.

Re:Llama and ChatGPT Are Not Open-Source by kristopolous on Friday July 28, 2023 @01:37AM (#36903409)
An grammar so no because no rules have then its they're simple she great understand your of problem, yes?

Or did I break some rules there?

Re:Llama and ChatGPT Are Not Open-Source by reverius42 on Friday July 28, 2023 @04:25AM (#36904272)
It was more the correct spelling claim I object to. Considering all variations of English.
Re:Llama and ChatGPT Are Not Open-Source by kristopolous on Friday July 28, 2023 @09:18AM (#36906350)
The point stands. Things are often depicted too binary and simplistic. Many subjective things also get smuggled in as if they were objective to hide their ideology and the word "truth" is used to stop people from questioning things.

I'm totally onboard with that.

At the same time, when talking about the physical world, most things are not like that. Humans for instance, get around by moving limbs called feet and not flapping feathered wings like a bird. This is the level of obvious I'm talking about.

The secondary point was that many things are this level of obvious and also specialized. For example: how does the animal cacomantis castaneiventris get around? Unless you have some kind of specialized knowledge you wouldn't know that's a type of bird.

The internet is awash in unspecialized people opining on specialized knowledge that is only controversial to the untrained eye in the same way that students don't get As on every exam they take. You need to actively work around people talking out of their bounds if you want your training data to be accurate

Re:Llama and ChatGPT Are Not Open-Source by nomel on Thursday July 27, 2023 @07:16PM (#36901348)
> then it gives people leverage to neuter it

Could you expand on that a bit?

Re:Llama and ChatGPT Are Not Open-Source by goatlover on Thursday July 27, 2023 @07:18PM (#36901363)
Add me to the list. I share your opinion.
Re:Llama and ChatGPT Are Not Open-Source by barbariangrunge on Thursday July 27, 2023 @07:51PM (#36901608)
> Copyrighted material, sexual content, political opinions, throw it all in and release it please!

Why copyrighted material? Could we stop celebrating how tech is going to steal everyone's copyrighted works in a massive effort to replace the artists who made it? Why does everyone here hate artists so much? Do they not deserve any rights over their IP, eg, the right to say no when someone wants to make derivative works that replace them from it?

Re:Llama and ChatGPT Are Not Open-Source by bick_nyers on Thursday July 27, 2023 @08:16PM (#36901776)
Pretty much everything nowadays is copyrighted, by omitting such materials, what are you really left with?

LLM is a tool much like the internet is a tool. Yes, someone can use it to steal, but stealing is against the law.

Instead of encoding a criminal justice system into an LLM by omitting the possibility of stealing an artists work or omitting the knowledge of physics so someone can't learn how to build a bomb, we should instead just prosecute people for using it in that way intentionally.

How often do people get prosecuted for ripping off an artists style? The criminal justice system "hated artists" long before LLMs and it's not the responsibility of the tech. companies to rectify that in my opinion.

Re:Llama and ChatGPT Are Not Open-Source by __loam on Thursday July 27, 2023 @08:32PM (#36901903)
You're not addressing the massive abuse of the commons this still represents. If artists don't have the right to tell you to fuck off for using their work in training data, they're less likely to publicly show that work, which hurts them because they become less visible and hurts the AI because the training data gets worse.
Re:Llama and ChatGPT Are Not Open-Source by smoldesu on Thursday July 27, 2023 @08:43PM (#36901969)
They do have an option; they can choose to not publish their work. By digitizing your creation, you are creating a version of your work that can be distributed at practically-free prices with little effort. If that undermines or undervalues the art you make enough, you can choose not to share it.

Even Open Source advocates don't really have the right to stop companies from using Open code. The license discourages it, but everyone from Tesla to Nintendo has been caught violating it's terms. Publishing stuff on the open web has always had consequences, unfortunately.

Re:Llama and ChatGPT Are Not Open-Source by Kuinox on Thursday July 27, 2023 @09:29PM (#36902188)
Go on youtube and type "copy arstyle". Now tell me how artists were not stealing from each other.
Re:Llama and ChatGPT Are Not Open-Source by __loam on Thursday July 27, 2023 @10:01PM (#36902385)
Artists are generally pretty encouraging to people entering the field and using their stuff as reference for new artists. That actually contributes to art. It's definitely not the same thing as a massive tech corporation trying to automate their livelihoods, but please keep making this flawed argument analogizing two completely different processes.
Re:Llama and ChatGPT Are Not Open-Source by Kuinox on Friday July 28, 2023 @05:46AM (#36904738)
> It's definitely not the same thing as a massive tech corporation trying to automate their livelihoods, but please keep making this flawed argument analogizing two completely different processes.

Ah yes, the famous massive tech corporation behind stable diffusion.

Re:Llama and ChatGPT Are Not Open-Source by dns_snek on Friday July 28, 2023 @05:02AM (#36904484)
> Now tell me how artists were not stealing from each other.

Key words being "from each other". AI only takes, it doesn't give anything back, it doesn't inspire. Their ultimate goal is to absorb the entire human history of art and then displace millions of people who received nothing from this transaction they were forced into, just so that some billionaire can afford another yacht.

I have no problems with AI models, but if you want to use art, writing, code, etc. for training, you should restrict your use to public domain works, ask for consent, or commission it. Obviously that would cost a lot of money so big tech companies are once again looking for a free ride.

Re:Llama and ChatGPT Are Not Open-Source by Kuinox on Friday July 28, 2023 @05:20AM (#36904587)
> AI only takes, it doesn't give anything back, it doesn't inspire.

Totally wrong. There is even a well known counter example: dream-like videos generated by ai wasn't something we had before. This statement tells me you never used it.

> just so that some billionaire can afford another yacht

You are heavily mistaken, that the scenario where ai learning isn't considered as fair use of material. In this case, only megacorps will be able to trains their own AI by spending billions in content.

> Obviously that would cost a lot of money so big tech companies are once again looking for a free ride.

You are constructing your own story while ignoring what is happening.

Adobe and Dall-e models were trained with datasets they mostly had rights over. OpenAI partnered with Shutterstock, and Adobe have their own photo stock. Big tech companies didn't had problem looking for image content. Stable Diffusion on the other hand is open source & open research, but didn't had any rights on most of the image they trained.

Re:Llama and ChatGPT Are Not Open-Source by LightHugger on Thursday July 27, 2023 @08:16PM (#36901774)
Modern copyright is not a good, it's an evil.
Re:Llama and ChatGPT Are Not Open-Source by threeseed on Thursday July 27, 2023 @08:28PM (#36901876)
Authors, artists, actors etc whose work have been ripped off by mega corporations like Meta, OpenAI, Google etc would disagree.

Copyright may need to be updated to cater for the new world of AI but that doesn't mean as a concept it is evil.

Re:Llama and ChatGPT Are Not Open-Source by murki on Thursday July 27, 2023 @08:39PM (#36901951)
How are Modern Copyright laws helping authors, artists, and actors in those cases?
Re:Llama and ChatGPT Are Not Open-Source by threeseed on Thursday July 27, 2023 @09:11PM (#36902085)
It's allowing them to sue OpenAI for copyright infringement:

https://www.theguardian.com/books/2023/jul/05/authors-file-a...

Re:Llama and ChatGPT Are Not Open-Source by ronsor on Thursday July 27, 2023 @09:41PM (#36902253)
* * *
Re:Llama and ChatGPT Are Not Open-Source by CamperBob2 on Thursday July 27, 2023 @09:54PM (#36902332)
Allowing them to block human intellectual progress may not be the long-term win you assume it is.
Re:Llama and ChatGPT Are Not Open-Source by threeseed on Thursday July 27, 2023 @11:49PM (#36902955)
And some would argue that billion dollar companies like OpenAI, Meta etc should figure out a way to compensate the people who created the content they depend on.
Re:Llama and ChatGPT Are Not Open-Source by CamperBob2 on Friday July 28, 2023 @12:09PM (#36909005)
Making the uncensored, unbiased, un-"aligned" results available to all for a fair price is compensation enough, I think. That's what we should be pushing for.

Whether demon or genie, any notion of putting this tech back in the bottle is a non-starter. As is turning it into a giant money grab for the copyright industry.

Re:Llama and ChatGPT Are Not Open-Source by smoldesu on Thursday July 27, 2023 @11:58PM (#36902993)
That hasn't protected copywritten works in the past: https://en.wikipedia.org/wiki/Authors_Guild,_Inc._v._Google,....
Re:Llama and ChatGPT Are Not Open-Source by Andrex on Thursday July 27, 2023 @08:23PM (#36901834)
That depends on the jurisdiction, but in my view I err on the side of "but it's necessary."

A society without copyright would be a much poorer one, and humans figured that out a long time ago.

Re:Llama and ChatGPT Are Not Open-Source by nightski on Thursday July 27, 2023 @08:25PM (#36901850)
I don't think you can look to the past to picture a future with AI.
Re:Llama and ChatGPT Are Not Open-Source by CamperBob2 on Thursday July 27, 2023 @09:54PM (#36902339)
A society without copyright would be a much poorer one

That was before "Attention Is All You Need" and the subsequent work on LLMs that grew out of it. Things are different now.

Re:Llama and ChatGPT Are Not Open-Source by __loam on Thursday July 27, 2023 @08:32PM (#36901908)
People should be paid for their labor.
Re:Llama and ChatGPT Are Not Open-Source by smoldesu on Thursday July 27, 2023 @10:11PM (#36902450)
Then they shouldn't give away their work for free.
Re:Llama and ChatGPT Are Not Open-Source by __loam on Friday July 28, 2023 @12:30AM (#36903123)
They're not. They have a copyright. Practicing artists usually benefit professionally from maintaining a public portfolio. Data being public is also notably not a license to use it for whatever purpose you want. Have some respect.
Re:Llama and ChatGPT Are Not Open-Source by smoldesu on Friday July 28, 2023 @12:59AM (#36903239)
I have a copyleft, for all the good it does my work. Copyright protects artists from nothing, and leaving it uncontested harms the consumer more than the artists.

> Data being public is also notably not a license to use it for whatever purpose you want.

Under the Fair Use doctrine, it very well could be. It was when Google indexed every book they could buy, to the disdain of the Author's Guild and the titleholders they represented.

> Have some respect.

I will not respect an authority that forces me to rent digital content.

Re:Llama and ChatGPT Are Not Open-Source by hmcq6 on Friday July 28, 2023 @01:28AM (#36903367)
> I will not respect an authority that forces me to rent digital content.

They are very obviously asking you to respect the individual artists.

Re:Llama and ChatGPT Are Not Open-Source by philipov on Thursday July 27, 2023 @08:24PM (#36901842)
We know who copyright is meant to protect, and it's not artists: it's Disney's shareholders. Pay writers and actors a fair cut, and then maybe people will take seriously that copyright is for artists.
Re:Llama and ChatGPT Are Not Open-Source by threeseed on Thursday July 27, 2023 @08:30PM (#36901890)
It's both. Because we don't live in a child-like, black & white world.

Copyright like many regulatory devices can be used for good, evil and everything in between.

Re:Llama and ChatGPT Are Not Open-Source by accrual on Thursday July 27, 2023 @08:03PM (#36901702)
I wonder if there's a future title like "AI Model Artist", "AI Model Contributor", "AI Model Author", "AI Model Creator" or something for creators who contribute to the models in use.

While I love we can all immediately benefit from previous works, I feel for the countless artists and creators who's work has been integrated into models with zero compensation. There are already outstanding lawsuits seeking reparations, but this is new legal territory and there will be a lot to learn and decide in the future.

Re:Llama and ChatGPT Are Not Open-Source by __loam on Thursday July 27, 2023 @08:34PM (#36901915)
I think if AI art damages the commons to an extreme extent, there's a real possibility that AI companies pay artists to make whatever the fuck they want to keep the training data coming.
Re:Llama and ChatGPT Are Not Open-Source by paul_funyun on Thursday July 27, 2023 @07:57PM (#36901656)
I can't imagine what goes through the head of someone who thinks they don't have the right to say no. They don't have the right to be listened to or the right to use state power to enforce a colloquial notion of IP and rights with no basis in law or fact, but they surely have the right to say "No!"

In all seriousness, they're afraid because they aren't very pleasant. AI will write things people will like and enjoy hearing. Preachy artists these days just make works they think people ought to want, and ought to like. Getting moralism out of art will be a great technological accomplishment.

Re:Llama and ChatGPT Are Not Open-Source by eropple on Thursday July 27, 2023 @08:20PM (#36901813)
Moralism! In art! Heaven forfend. How dare that art have a message, right? Never in history, not once, before the Woke Tens and, I guess, the Woker Twenties did somebody have an opinion and encode it in allegory. It is a purely novel phenomenon and all known media before this dark age was absolutely without message and without any editorial lens to determine what is and is not to be depicted--for that, of course, would communicate values and ideas, and art simply did not do such a thing.

More seriously: if you have this attitude for-reals and aren't Poe's Lawing me, perhaps you need a little more preaching aimed your way because your conception of art and the civic society is the closest thing I can think of to a humanities position being objectively wrong.

And the ideological soma of agree-with-you AI being desirable is some apocalyptic stuff.

Re:Llama and ChatGPT Are Not Open-Source by CatWChainsaw on Thursday August 03, 2023 @09:42AM (#36985787)
I want to see the world where this happens, where everything that made life enjoyable is corporatized, standardized, optimized, one-dimensionalized. Since it will happen over the course of your lifetime, you'll get to see it happen - as things that were fulfilling cease to be, an ennui you can't escape sets in because you built every wall of your cage one by one, and all you can do is regret. And then the Black Mirror helmet comes off your head, everyone else points and laughs at you, and then puts the helmet back on.
Re:Llama and ChatGPT Are Not Open-Source by ineedasername on Thursday July 27, 2023 @08:47PM (#36901979)
>Could we stop celebrating how tech is going to steal everyone's copyrighted works

Copyright has a fair-use exception for transformative works. It is difficult to look at the LLM's of the day and think "No, they have not taken the copywritten works and transformed them into something completely new."

There is no hate of artists. I don't know how you get from "It's not copyright violation" to "I hate artists". This is a matter of existing law. It is of course unsettled, the fair-use interpretation not yet tested in courts. But the same goes for affirming it's a violation of copyright.

These questions will have their day in court, and until then there is no need to make arguments in an inflammatory tone that that borders on personal attack. In the meantime, maybe engage in a conversation about how copyright law would need to be changed to account for this new technology, not condemn people who see a reasonable interpretation of existing law the differs from your own.

Re:Llama and ChatGPT Are Not Open-Source by reverius42 on Thursday July 27, 2023 @10:21PM (#36902512)
So I can have a giant database of all the most recent copyrighted works, on my home computer, as long as I claim I'm using it for training a model? And if I happen to listen to some music or watch some movies too, who's going to know?

I think an argument could be made that it's fine for an LLM to learn from copyrighted works, but maybe it should have to go to the library to do it. Having your own human-accessible copy of those works (at home, or at Meta), doesn't sound as acceptable to me.

Re:Llama and ChatGPT Are Not Open-Source by ineedasername on Thursday July 27, 2023 @10:36PM (#36902596)
Sure, I think you’re right, it’s reasonable to expect a company to at least buy a single copy of each book it ingests if the work could not otherwise be legally obtained for free. Or something reasonable along those lines
Re:Llama and ChatGPT Are Not Open-Source by NLPaep on Friday July 28, 2023 @03:44AM (#36904052)
Yes, let’s shut out the small players now go Microsoft can bilk us
Re:Llama and ChatGPT Are Not Open-Source by ineedasername on Friday July 28, 2023 @07:30AM (#36905385)
It’s not Microsoft’s fault that people are generally required to buy a book in order to read it. You could say the same thing about any business with a high capital cost required to get up and running. Want to start a supermarket? That’s many $Millions to plan, buy/rent land, build the facility, stock the shelves, etc. Starting any business costs money, and some cost a lot of money. Suppliers of the materials or feedstock required to start them aren’t to blame. If I want to fine tune a LLM to be an expert on the Dune book series, should I be allowed to download a copy without paying, or should I have to pay $8 for a copy before I feed it to the model? I’m sure the publisher would want me to pay some ridiculous amount but that is a different argument than requiring the purchase of a single copy that anyone would have to do to get access to the content.
Re:Llama and ChatGPT Are Not Open-Source by madsmith on Thursday July 27, 2023 @08:53PM (#36902014)
I believe LLMs should be allowed to read/view/consume content and learn from it even if that content has a copyright.

We phrase it like somehow the material is being copied into the LLM, but that’s not what it’s doing. It’s building a neural graph from the experience of consuming that content.

What would the world be like if humans couldn’t learn, train the weights of the interconnects of their neural tissue, from any material with a copyright?

Re:Llama and ChatGPT Are Not Open-Source by api on Thursday July 27, 2023 @09:00PM (#36902037)
It’s a form of lossy compression. Can I strip the copyright off an image by JPEG compressing it?

At the very least I think LLMs trained on data that the trainer does not own or have rights to use in that manner should not be copyrightable.

Re:Llama and ChatGPT Are Not Open-Source by bick_nyers on Thursday July 27, 2023 @09:13PM (#36902093)
An LLM is a lossy compression of the internet and I think it should be treated as such. You can't copyright the internet itself.
Re:Llama and ChatGPT Are Not Open-Source by madsmith on Thursday July 27, 2023 @11:03PM (#36902713)
All knowledge is lossy compression.

My thinking “the enemy gate is down” when considering the tokens “Ender’s Game” is my recalling a learned association of those tokens to the given token string.

My knowing that doesn’t strip the copyright. My telling someone the meaning and context of the phrase generally doesn’t strip the copyright away from Orson Scott Card. I’m not reproducing his work but my knowledge of it. And it’s dependent on what I do with that knowledge and how if I’ve violated his copyright.

We are prosecuting the LLMs for possessing fragments of knowledge. And we’re assuming that the recall of some of those fragments means a copy of that work is in fact contained within the weights.

Re:Llama and ChatGPT Are Not Open-Source by hmcq6 on Friday July 28, 2023 @01:26AM (#36903356)
A mathematical transformation of the data is not enough to qualify as a transformative work. Saving a copy-written work in a lossy compression does not negate the copywrite.
Re:Llama and ChatGPT Are Not Open-Source by Kuinox on Thursday July 27, 2023 @09:26PM (#36902163)
There is a lot of mental gymnastic, to submit an artwork publicly on the internet, allowing everyone to copy your arstyle, make derivative art of it, have other learn from it, but if ever a machine 'learn' from it, it's "stealing". It's not because you don't like a derivative work, that the derivative work is "stealing" your content. Saying it's stealing is wrong, it's lying to get your point accross.

You blame "how tech is going to steal everyone copyrighted works", yet, they were already in the tech world: the internet.

Re:Llama and ChatGPT Are Not Open-Source by raincole on Thursday July 27, 2023 @09:43PM (#36902266)
It's a lot of mental gymnastic to think machine learning = human learning. Especially on HN, where people should understand scale matters a lot in real world.

It's generally consider ok to sell your fanart on comic festivals. But do you think it's okay that Disney starts selling fanart of One Piece without the publisher's permission?

A car and a pair of legs both move you from A point to B point. So why do laws treat automobiles and pedestrians so differently?

Re:Llama and ChatGPT Are Not Open-Source by Kuinox on Friday July 28, 2023 @05:38AM (#36904695)
> It's a lot of mental gymnastic to think machine learning = human learning.

That's why I have put it in quotes. > But do you think it's okay that Disney starts selling fanart of One Piece without the publisher's permission?

It's why it's called "fair use". Fanart isn't the only case of fair use. Parody for example allows it. And AFAIK, no court yet have decided that AI "learning" is not fair use or what the limite are.

Re:Llama and ChatGPT Are Not Open-Source by hmcq6 on Friday July 28, 2023 @02:35AM (#36903670)
> have other learn from it, but if ever a machine 'learn'

Why did you put one of those in quotes and not the other?

Re:Llama and ChatGPT Are Not Open-Source by Kuinox on Friday July 28, 2023 @05:45AM (#36904732)
Because these two are similar, yet not the same.
Re:Llama and ChatGPT Are Not Open-Source by accrual on Thursday July 27, 2023 @08:07PM (#36901725)
> Yes, reducing bias in the models is a noble goal, but introducing new bias and blindspots to do it is a no-no.

What other option would there be though? It seems like a binary field. You can either leave it wholly unaligned, or you can attempt to align the model, and you will introduce your own bias as a side effect.

In the future, and in fact in the present, there is already a "gray market" for unaligned models. There will surely be market for these and they will sell just any other item on this market.

Re:Llama and ChatGPT Are Not Open-Source by dragonwriter on Thursday July 27, 2023 @08:12PM (#36901744)
> Yes, reducing bias in the models is a noble goal

“Bias” is not a unidimensional thing (or even one with a meaningful magnitude); you can’t reduce bias, you can make it more transparent (better documented) so that you can evaluate appropriateness for particular uses.

While I am not a big fan of the crew that pushes “alignment” as the main issue with regard to AI for other reasons, that’s a particularly good term for what is important – aligning the biases of the AI with the usage intent. “Reducing” bias makes sense only with regard to a specific usage intent.