If you forbid it, the investment in developing those models disappears. They will be stuck at ~what we have now at best.

You can also require cloud providers to enforce a ban on training (and deploying) such models, it's doable. Good luck training it in your basement, it will probably take you a decade.

If this is banned, it will become a lot like piracy - yes, it's available, no, most people (at least in the West) don't do it, practically no businesses do it.

Training these models is much much cheaper than you think it is, and there’s good data for it already.

Either use a CC0 set like Wikimedia/Flickr and throw in some dead artists like Brueghel, or train on data from a country we don’t respect the IP of. Lots of Taobao product photos out there. It’s enough.

You are getting ridiculous now. Training this on "Taobao product photos" will lead to a useless model that is unable to produce practically all of the "cool" demos posted here in the last week.

A few months ago this task was virtually impossible. Then it was possible, but extremely expensive and pay-walled behind the "Open"AI' website.

As of about a week ago this tech runs on consumer GPUs. The weights have been downloaded 100s of thousands of times, and fine-tuning / modifying is possible.

Training from scratch is about $500k still, but it will only get cheaper and easier.

This doesn't contradict anything I have written. The average technical user will be unable to train this exact model (not to mention the supposed future more powerful ones) in their basement in this decade.

That just feels like such a pessimistic forecast to me. Of course, the current trajectory of improvements in model efficiency and better commercial GPUs / ML-accelerators may hit a wall.

But I would not be surprised if this was trainable on a commercial GPU at home within that time. But I think another important trend that we are seeing is that you don't need to train these models from scratch.

Open-source "foundation models" means that you can usually get away with the much easier task of fine-tuning, as to not throw away / re-learn everything that these large models have already fit.

Edit: I initially said 2-5 years, but on more reflection this does seem optimistic (for training from scratch).

If things go that way, the 'legit' models will continue dev, just using licensed content (along with public domain works). It will be more expensive for the end user, but that cost will shrink over time for general work. Tools that mimic working artist though might not be available (or will expensive). This all seems pretty ideal, so the pessimist in me guesses it's fairly unlikely.

I am not sure it will be possible to get enough training data that way.

I don't know enough about diffusion models but if LLMs (of current size) have to use only public domain, they will be undertrained and we will see significant degradation in performance. Not to mention that Codex will be effectively dead.