Who will also win: Google, OpenAI and other corporations that enter exclusive deals, that can more and more rely on synthetic data, that can build anti-recitation systems, etc.
And of course the lawyers. The lawyers always win.
Who will not win:
Millions of independent bloggers (whose content will be used)
Millions of open source software engineers (whose content will be used against the licenses, and used to displace their livelihood), etc.
The likes of Google and OpenAI entered the space by building on top of the work of the above two groups. Now they want to pull up the ladder. We shouldn't allow that to happen.
They haven't been stolen; the creators still have them. They've just been copied. It's amazing how much the ethos on this site has shifted over the past decade, away from the hacker idea that "intellectual property" isn't real property, just a means of growing corporate power, and information wants to be free.
You wouldn't download a car.
Sure you might not like OpenAI, but maybe some other company comes a long and builds the next magical product using information that is freely available.
I don't care if employees at OpenAI read books from their local library on python. More power to them. I don't even care if they copy the book for reference at work, still fine. But utilizing language at scale as a scarce resource to train models is not that and is not in any way analogous to it.
Public information should should be free for anyone to consume and use how they want.
A truly hilarious sentiment coming from someone making zero effort to actually engage with what I'm saying in favor of parroting back empty platitudes.
It hasn't. The hacker ethos is about openness, individuality, decentralization (among others).
OpenAI is open in what it consumes, not what it outputs.
It makes sense to have protections in place when your other values are threatened.
If "information want's to be free" leads to OpenAI centralizing control over the most advanced AI then will it be worth it?
A solution here would be similar to the GPL: even megacorps can use GPL software, but they have to contribute back. If OpenAI and the rest would be forced to make everything public (if it's trained on open data) then that would be an acceptable compromise.
Yes, the greatest things on the internet have been decentralized - Git, Linux, Wikipedia, open scientific publications, even some forums. We used to passively consume content and internet allowed interaction. We don't want to return to the old days. AI falls into the decentralized camp, the primary beneficiaries are not the providers but the users. We get help of things we need, OpenAI gets a few cents per million tokens, they don't even break even.
I'm sorry, the worlds knowledge now largely accessible by a laymen via LLMs controlled by at most, 5 companies is decentralized? If that statement is true then the world decentralized truly is entirely devoid of meaning at this point.
1. Decentralized technologies you can operate privately, freely, and adapt to your needs: computers, old internet, Linux, git, FireFox, local Wikipedia dump, old standalone games.
2. Centralized technologies that invade privacy, lead to loss of control and manipulation: web search, social networks, mobile phones, Chrome, recent internet, networked games. LLMs fall into the decentralized camp.
You can download a LLM, run it locally, fine-tune it. It is interactive, the most interactive decentralized tech since standalone games.
If you object that LLMs are mostly centralized today (upfront cost of pre-training and OpenAI popularity), I say they are still not monopolies, there are many more LLM providers than search engines and social networks, and the next round of phones and laptops will be capable of local gen-AI. The experience will be seamless, probably easier to adapt than touchscreens were in 2007.
I feel like the argument is akin to some countries considering rubbish, the things you throw away, to still be owned by your person ie "dumpster diving" is theft.
If a company had scraped public posts on the Internet and used it to compile art by colourising chunks of the text, is it theft? If an individual does it, is it theft?
If you as a developer look at how one would declare a function in python, review a few examples, you now know how to do that. Copilot can't say the same. It needs to see dozens, hundreds, perhaps thousands of them to reasonably accurately be counted on to accomplish that task, it's just how the tech works. Ergo, scaled data sets that can accomplish this teaching task now have value, if the people doing that training are working for high-valuation startups with the objective of selling access to code generating robots.
I'm all for abolishing copyright, for everyone. Let the knowledge be free and widely shared.
But until that is the case and people running super useful services like libgen have to keep hiding then I also want all the LLM corpos to be subject to the same legal penalties.
LLM tech is pretty cool.
Would be a lot cooler if its existence wasn’t predicted on the wholesale theft of everyone’s stuff, immediately followed by denial of theft, poisoning the well, and massively profiting off it.
And LLMs are really a bad choice for infringement. They are slow, costly and unreliable at replicating any large piece of text compared to illegal copying. There is no space to perfectly memorize the majority of its training set. A 10B models is trained on 10T tokens, no space for more than 0.1% to be properly memorized.
I see this overreaction as an attempt to strengthen copyright, a kind of nimby-ism where existing authors cut the ladder to the next generation by walling off abstract ideas and making it more probably to get sued for accidental similarities.
People who confidently assert either opinion in this regard are wrong. The lawsuits are still pending. But if I had to bet, I'd bet on the OpenAI side. Even if they don't win outright, they'll probably carve out enough exemptions and mandatory licensing deals to be comfortable.
That profit will be squeezed to zero over the long term if Zuck maintains his current strategy.
That's not true anymore, Meta isn't behind OpenAI
If Meta stopped now, there would still be profit in the market, but if they keep releasing Llamas for the next 5+ years then OpenAI et al will be fighting for scraps. Not everybody needs a model that can prove theorems.
Pirate Bay: largely facilitating the theft of material from large corporations by normal people, for generally personal use.
LLM training: theft of material from literally _everyone_, for the purposes of corporate profit (or, well, heh, intended profit; of course all LLM-based enterprises are currently massively loss-making, and may remain so forever).
This undermines your own point.
Also, open source models exist.
Help me understand the negative effect of AI and LLMs on hackers.
I've never actually used an LLM though - I just don't have any use for such a thing. All my writing and programming are done for fun and automation would take that away.
Bill Gates' infamous letter was sent in 1976[1].
[1]: https://en.wikipedia.org/wiki/An_Open_Letter_to_Hobbyists
So, capitalism.
It's taboo to criticize that though.
In what world is this taboo? That critique comes back in at least half the HN threads about AI.
Watch any non-technical video about AI on Youtube and it will mention people being worried of the power of mega-corporations.
Your take is about as taboo as wearing a Che Guevara tshirt.
All sorts of taboo stuff gets said on hacker news. It's not exactly a cultural bellwether that something is allowed to be said here.
Wearing a che guevara t shirt is similarly "allowed", but the last article about him in the new york times is fawning administration for his assassin.
Criticism of capitalism is taboo.
It's not, that's playing the victim. There are hundreds or thousands of posts daily all over HN criticising capitalism. And most seem upvoted, not downvoted.
Don't even get me started on reddit.
However markets also have limits. A power company competing for your business is largely a farce, since the power lines to your home will not change. A cable company in America is almost certainly a functional monopoly, and that fact is reflected in their quality of service. Infrastructure of all sorts makes for piss-put markets because true competition is all but impossible, and even if it does kind of work, it’s inefficient. A customer must become knowledgeable in some way to have a ghost of a clue what they’re buying, or trust entirely dubious information from marketing. And, even if somehow everything is working up to this point, corporations are, above all, cost cutters and if you put one in charge of an area where it feels as though customers have few if any choices and the friction to change is high, they will immediately begin degrading their quality of services to save money in the budget.
And this is only from first principles, we have so many other things that could be discussed from mass market manipulation to the generous subsidies of a stunning variety that basically every business at scale enjoys to the rapacious compensation schemes that have become entirely too commonplace in the executive room, etc etc etc.
To put it short: I have no issue at all with capitalism operating in non-essential to life industries. My issue is all the ways it’s infiltrated the essential ones and made them demonstrably worse, less efficient, and more expensive for every consumer.
The competitive landscape where consumers vote for the best products with their purchasing decisions is simply not a sustainable equilibrium.
The ability to accumulate capital (I.e. “capitalism”) leads to regulatory protectionism through lobbying, bribery, etcetera.
The biggest and most salient critique of liberal capitalism as we now exist under is that it requires far too many of the "right people" to be in positions of power; it presumes good faith where it shouldn't, and fails to reckon with bad actors as what they are far too often, the modern American Republican party being an excellent example (but far from the only one).
There is a saying that when you have a hammer, every problem looks like a nail.
And you're trying to suggest that these two are the same?
Edit: I don't mind downvotes, karma means nothing, but I do appreciate when folk speak up and say why I might be wrong. :)
Use source-aware training, use the same datasets as used in LLM training + copyrighted content. Now the LLM can respond not just what it thinks is most likely but also what source document(s) provided specific content. Then you can consult commercially available LLM's and detect copy right infringements, and identify copyright holders. Extract perpetrators and victims at scale. To ensure indefinite exploitation only sue commercially succesful LLM providers, so there is a constant new flux of growing small LLM providers taking up the freed niche of large LLM providers being sued empty.
My understanding (as one uninvolved in the industry) is that this is more or less a completely unsolved problem.
https://github.com/mukhal/intrinsic-source-citation
The only 2 reasons big LLM providers refuse to do it is
1) to prevent a long slew of content creators filing class action suit.
2) to keep regulators in the dark of how feasible and actionable it would be, once regulators are aware they can perform the source-aware training themselves
People crying for copyright in the context of AI training don't understand what copyright is, how it works and when it applies.
What they think how copyright works: When you take someones work as inspiration then everything you produce form that counts as derivative work.
How copyright actually works: The input is irrelevant, only the output matters. Thus derivative work is what explicitly contains or resembles underlying work, no matter if it was actually based on that or it is just happenstance / coincidence.
Thus AI models are safe from copyright lawsuits as long as they filter out any output which comes too close to known material. Everything else is fine, even if the model was explicitly trained on commercial copyrighted material only.
In other words: The concept of intellectual property is completely broken and that is old news.
Scraping websites containing source code which is distributed with specific licenses that OpenAI & co don't follow.
One can train a model exclusively on source code from the linux kernel (GPL) and then generate a bunch of C programs or libraries from that. And they could publish them under MIT license as long as they don't reproduce any identifiable sections from the linux kernel. It does not matter where the model learned how to program.
If I write code with a license that says that using this code for AI training is forbidden then OpenAI is directly going against this by scraping websites indiscriminately.
There is a reason why it is generally advised to go with the established licenses and not invent your own, similarly to how you should not roll your own cryptography: Because it most likely won't work as intended.
e.g. License: This comment is licensed under my custom L*a license. Any user with an username starting with "L" and ending in "a" is forbidden from reading my comment and producing replies based on what I have written.
... see?
Sorry, but the onus is on OpenAI to read the licenses not the creator.
And throwing your hands in the air and saying "oh you can't do that in a license" is also of little use.
A license (copyright law) is not a contract (contract law). Simply publishing something does not make the whole world enter into a contract with you. Others first have to explicitly agree to do so.
> Sorry, but the onus is on OpenAI to read the licenses not the creator.
They can ignore it because they never agreed to it in the first place.
> And throwing your hands in the air and saying "oh you can't do that in a license" is also of little use.
It is very useful to know what works and what does not. That way you don't trick yourself and your work to be safe, don't get caught by surprise if you are in fact not and can think of alternatives instead.
BTW, a thing you can do (which CaptainFever mentioned) and lots of services do because licenses are so weak is to make people sign up with an account and have them enter a ToS agreement instead.
They did by accessing and copying the code. Same as a human cloning a repository and using it's content or someone accessing a website with Terms of Use.
No signed contract is needed here.
By default, copying is disallowed because of copyright. Your license provides them a right to copy the code, perhaps within certain restrictions.
However, sometimes copying is allowed, such as fair use (I covered this in another comment I sent you). This would allow them to copy the code regardless of the license.
> Same as a human cloning a repository and using it's content or someone accessing a website with Terms of Use.
I've covered the cloning/copying part already, but "I agree to this ToS by continuing to browse this webpage" is called a clickwrap agreement. Its enforceability is dubious. I think the LinkedIn case showed that it only applied if HiQ actually explicitly agreed to it by signing up.
Now if the likes of Openai and Google keep the model weights private and just provide generated text, they can try to filter for derivative works, but I don't see a solution that doesn't leak. If a model can be coaxed into producing a derivative work that escapes the filter, then boom, unlicensed copy was provided. If I tell the model to mix two texts word by word, what filter could catch this? What if I tell the model to use a numerical encoding scheme? Or to translate into another language? For example assuming the model knows a bunch of NYT articles by heart, as was already demonstrated: If have it translate one of those articles to French for me, that's still an unlicensed copy!
I can see how they will try to get these violations legalized like the DMCA safe-harbored things, but at the moment they are the ones generating the unlicensed versions and publishing them when prompted to do so.
> People crying for copyright in the context of AI training don't understand what copyright is, how it works and when it applies.
People are complaining about what's happening, not with the exact wording of the law.
What they are doing probably isn't illegal, but it _should_ be. The problem is that it's very difficult for people to pass new legislation because they don't have lobbyists the way corporations do.
The things that constitute potentially infringing copying are not clearly well-defined, and whether or not training an AI is on that list has of course not yet been considered by a court. But you can make cogent arguments either way, and I would not be prepared to bet on either outcome. Keep in mind also that, legally, copying data from disk to RAM is considered potentially infringing, which should give you a sense of the sort of banana-pants setup that copyright can entail.
That said, if training is potentially infringing on copyright, it now seems pretty clear that a fair use defense is going to fail. The recent Warhol decision rather destroys any hope that it might be considered "transformative", while the fact that the AI companies are now licensing content for training use is a concession that the fourth and usually most important factor (market impact) weighs against fair use.
But you are right, we don't know until president is set by a court. I am only warning people that laying back and hoping that copyright will apply as they wish is not a good strategy to defend your work. One should consider alternative legal constructs or simply not releasing material to the general public anymore.
From the context of the comment it was pretty clear that they were using theft as shorthand for taking without permission.
If I have something, and you copy it, then I still have that thing.
The point of anti-copyright advocates are that merely copying is not ethically wrong. In fact, Why Software Must Be Free made the argument that preventing people from copying is ethically wrong because it limited the spread of culture and reuse.
That is the crux of the disagreement. You may rephrase our argument as "physical theft may be bad, but digital theft is not bad, and in fact preventing digital theft is in itself bad", but the argument does not change.
Of course, there is additional disagreement in the implied moral value of the word "theft". In that case I agree with you that pro-copyright/anti-AI advocates have made their point by the usage of that word. Of course, we disagree, but... it is what it is I suppose.
Copyright as far as I understand is focused on wholesale reproduction/distribution of works, rather than using material for generation of new works.
If something is available without contractual restriction it is available to all. Whether it's me reading a book, or a LLM reading a book, both could be considered the same.
Where the law might have something to say is around the output of said trained models, this might be interesting to see given the potential of small-scale outputs. i.e. If I output something to a small number of people, how does one detect/report that level of infringement. Does the `potential` of infringement start to matter.
The problem (as far as this specific discussion goes) is not that IP laws exist, but rather that they are only being applied in one direction.
Not necessarily. An alternative could be to say that all models trained on data which hasn't been explicitly licensed for AI-training should be made public.
I believe that all the copyright suits against AI companies will be total failures because I can't come up with a answer to any of those questions.
IP has had ample support because the "protect the little artist" argument is compelling, but it is just not how the world works.
That's factually wrong. Capitalism is about moving wealth more efficiently: easier to allocate money/wealth to X through the banking system than to move sheep/wealth to X's farm.
> Capitalism is about moving wealth more efficiently: easier to allocate money/wealth to X through the banking system than to move sheep/wealth to X's farm.
It's not. That's what money's about. Any system with an abstract concept of money admits that it's easier to allocate wealth with abstractions than physically moving objects.
Capitalism is about capital. It's an economic system that says individuals should own things (i.e. control their purpose) by investing money (capital) into them. You attempted to correct the previous commenter, but provided an incorrect definition. I hope that clears up the relevance issue for you.
Yes. It's not about stealing land and kicking people out and raising sheep there instead. That (stealing) happens of course but is totally independent from any capitalist system.
JFC, the same sentence could have been said with communism in mind.
> You attempted to correct the previous commenter, but provided an incorrect definition. I hope that clears up the relevance issue for you.
You are confusing the intent of capitalism - which I gave the general direction of - with its definition. Does that clear up the relevance issue for you ? Did I fucking not write wealth/money intentionally ?
You didn't describe the intent or the definition of capitalism. You just mentioned that moving resources from A to B is easier if they're abstract. The intent of capitalism is that the society is improved when people who have a vested interest (literally vested, as in with money) in a company determine its actions.
Can you calm down, please? You were the one who started saying my comment was irrelevant. I don't think it makes much sense to parrot that back to me.
We should just scrap copyright and everybody plays a fair game, including us hackers.
Sue me because of breach of contract in civil court for damages because I shared your content, don't send the police and get me jailed directly.
I had my software cracked and stolen and I would never go after the users. They don't have any contract with me. They downloaded some bytes from the internet and used it. Finding whoever shared the code without authorization is hard and even so, suing them would cost me more than the money I'm likely to get back. Fair game, you won.
It's a natural market "tax" on selling a lot of copies and earning passively.
Basically you start with a stone in a pot of water — a neural net technology that does nothing meaningful but looks interesting. You say: "the soup is almost done, but would taste better given a bunch of training data." So you add a bunch of well curated docs. "Yeah, that helps but how about adding a bunch more". So you insert some blogs, copy righted materials, scraped pictures, reddit, and stack exchange. And then you ask users to interact with the models to fine tune it, contribute priming to make the output look as convincing as possible.
Then everyone marvels at your awesome LLM — a simple algorithm. How wonderful, this soup tastes given that the only ingredients are stones and water.
From Wikipedia:
> Some travelers come to a village, carrying nothing more than an empty cooking pot. Upon their arrival, the villagers are unwilling to share any of their food stores with the very hungry travelers. Then the travelers go to a stream and fill the pot with water, drop a large stone in it, and place it over a fire. One of the villagers becomes curious and asks what they are doing. The travelers answer that they are making "stone soup", which tastes wonderful and which they would be delighted to share with the villager, although it still needs a little bit of garnish, which they are missing, to improve the flavor.
> The villager, who anticipates enjoying a share of the soup, does not mind parting with a few carrots, so these are added to the soup. Another villager walks by, inquiring about the pot, and the travelers again mention their stone soup which has not yet reached its full potential. More and more villagers walk by, each adding another ingredient, like potatoes, onions, cabbages, peas, celery, tomatoes, sweetcorn, meat (like chicken, pork and beef), milk, butter, salt and pepper. Finally, the stone (being inedible) is removed from the pot, and a delicious and nourishing pot of soup is enjoyed by travelers and villagers alike. Although the travelers have thus tricked the villagers into sharing their food with them, they have successfully transformed it into a tasty meal which they share with the donors.
(Open source models exist.)
> a billion thefts
>The biggest theft
>what’s been stolen
I do like how the internet has suddenly acknowledged that pirating is theft and torrenting IS a criminal activity. To your point, I'd love to see a massive operation to arrest everyone who has downloaded copyrighted material illegal (aka stolen), for the public interest.
I suggest to apply the same to property law: make a photo and obtain instant and unlimited rights of use. – Things may change faster than we may imagine…
"output a 90 minute harry potter sequel to the final film starring the original actors plus Tom Hanks"
You want to own abstract ideas because AI can rephrase any specific expression. But that is antithetic to creativity.
How is it acceptable to allow these companies to steal all of this copyrighted data and then turn around and use it to enforce their copyrights in the most heavy-handed manner? The irony is unbelievable.
The thing I’m tired of is elites stealing everything under the sun to feed these models. So funny that copyright is important when it protects elites but not when a billion thefts are committed by LLM folks. Poor incentives for creators to create stuff if it just gets stolen and replicated by AI.
I’m hungry for more lawsuits. The biggest theft in human history by these gang of thieves should be held to account. I want a waterfall of lawsuits to take back what’s been stolen. It’s in the public’s interest to see this happen.