News for hackers. Stuff that matters.
Re:Imagen, a text-to-image diffusion model by Mockapapella on Monday May 23, 2022 @05:27PM (#31484960)
Transformers are parallelize-able, right? What’s stopping a large group of people from pooling their compute power together and working towards something like this? IIRC there were some crypto projects a while back that we’re trying to create something similar (golem?)
Re:Imagen, a text-to-image diffusion model by joshcryer on Monday May 23, 2022 @05:52PM (#31485241)
There are people working on reproducing the models, see here for Dall-E 2 for example: https://github.com/lucidrains/DALLE2-pytorch

It's often not worth it to decentralize the computation of the trained model though but it's not hard to get donated cycles and groups are working on it. Don't fret because Google isn't releasing the API/code. They released the paper and that's all you need.

Re:Imagen, a text-to-image diffusion model by visarga on Monday May 23, 2022 @05:33PM (#31485029)
There are the Eleuther.ai and BigScience projects working on public foundation models. They have a few releases already and currently training GPT-3 sized models.
Re:Imagen, a text-to-image diffusion model by sinenomine on Monday May 23, 2022 @09:55PM (#31487182)
You really need a decent infiniband-linked cluster to train large models.