The main barrier to entry for them is the massive amount of refinement required to make any useful amount of the required materials, like multiple warehouses size operation.
---
The big difference here is stable diffusion has shown you can train a highly capable model under 1 million dollars. LLaMa is making some interesting advances as well (finetuning a LOrA takes ~3 hours on a 4090). Training/finetuning/inference capable AI is quickly becoming more feasible (albiet trailing the commercial stuff) for the average consumer hardware.
Can growth be too fast?
- development behind walled gardens and closed doors, which will accelerate the wealth gap to war- and poverty- inducing levels
- access to this powerful game changer for the people who would otherwise be losing their purpose in the social structure. There's no reason your job has to evaporate as an artist, or programmer, or anything else if you can just keep doing the same thing empowered by a cheap, accessible force multiplier like AI.
It really comes down to whether or not the closed models are _enough better_ that they can start charging exuberant rents to people so they can effectively continue contributing to society.
The paths I see, in broad strokes:
- models become open and easy to hack on. We all lose our jobs. But we can now find new jobs using these models to do similar work.
- models stay closed and only get developed by BigCo. We all lose our jobs. Remediation for this development may or may not happen and who knows what it will look like.
The core difference I think you and I are dancing around here is a belief in our collective ability to hit the brakes on progress. I understand the desire to do so, but you won't stop a river by putting a rock in the middle and I think this is a situation with that level of momentum.
Cheap general AI is a potential exploit of everything everywhere all the time. No one is prepared for it.
A billion eyes are powerful. Giving every actor an eye-making machine is uncharted territory. And opening it up is a one-way gate - you can always delay opening it up another day, but you can't un-open it once you've let it out. So you should be very, very certain you're prepared to commit yourself, everyone you know, and the entire human race to living in that world before you make it so.
The infosec world figured this out ages ago. Yes, it's possible that some bad actors will gain access to capabilities that they didn't have before - but the strength of a billion eyes is huge, and can rapidly accelerate solutions to deficits.
What we need is a massive, open, community curated dataset that continually evolves. Craft regulations that enforce data availability and free access. Without such mechanisms, AI systems all have a SPOF that's just waiting to bite the collective us in the ass.