There is little to no need for anyone to scrape the web themselves, CommonCrawl does it for free and provides access to all that data for free, they are a charity similar to The Internet Archive:
As for using that data to train a model, I've done it at my own company and it's not that expensive, certainly not the astronomically expensive figures people seem to think it is. Maybe a hobbyist can't do it for themselves, but my small company managed to get pretty good results training on our existing hardware. It's hard to give concrete figures because we manage our own hardware instead of using AWS, but all in we're talking on the order of 100k, as opposed to what I've heard people on HN think it costs, as if it's a $10 million+ enterprise.
Remember... Bits have color and AI models are a rainbow.
There are clearnet sites which host entire movies and games and there have been since the dawn of the internet, wouldn’t you expect those would be bigger targets to go after if the powers were there? All that to say, I’m not bullish on the future global leverage of American copyright law, especially when it comes to AI models.
Case in point: GDPR, you can vacuum as much private data as you wish from non-EU citizens, the moment you touch EU-citizen data you must abide by the GDPR or you don't have access to a market of 400+ million of some of the wealthiest countries in the world.
You don't need global jurisdiction when your economical power is large enough.
The real issue for machine learning is that it lets you build systems that optimize for some metric. That's what corporations do. I've been pointing out lately that the ethical problems for AI in the near term look very much like those for corporations.
There seems to be a lot of different ideas about 'quality'. A lot of people want a better-than-search tool that knows vast amounts and can be assigned tasks. I'm personally much more interested in a reasoning machine, I don't need it to include the entire contents of the New York Public Library.
I think the future you described is possible. I also think it's possible that corporations are overpaying for data that will become stale after several years and new fresh data is generated in such great quantity/quality that access to it is not capital intensive.
any law can be legislated, if it can be put down as text.
It may be considered unreasonable, but if the majority of people _do_ agree, then it could be done.
The law is simple... You have to have a license to the data to train a model with that data. If you don't... Get ready to be sued.
There's no good way to put the AI genie back in the bottle, or make sure it will not create catastrophic undesired consequences. Only way is forward, even if there is no good path in sight.
This won't be the case everywhere. UK has actually already explicitly ruled the opposite
To operate in the real world like humans you need to have motivation. What is an AI motivated by?
Seems like we almost need pre-ai scraped datasets, almost like how it’s sometimes very useful to have metals that were created before the first atom bombs (I don’t know too much about this, just that there’s a definite land in the radiated sand around that timeframe).
Once those laws are in place, the moat is dug - and large corporations with access to capital vacuum up good training data in bulk. The whole time loudly proclaiming that they're 'empowering creators' and 'rewarding original thinkers.' Now the large corporations have the largest and best datasets tied up and neat, deep moat. Any startup wanting to challenge the incumbents could have significantly better models, but without access to the same quality and volume of training data, they'll be at a massive disadvantage.