Re:CoLT5: Faster Long-Range Transformers With Conditional Co...
by xiphias2 on Monday March 20, 2023 @08:58PM
(#35241168)
The most interesting aspect of this research is that Google published it. They have a huge competitor in this area, it would make sense to stop publishing unless they think that OpenAI already does this.
[ Parent
| on HN ]
Re:CoLT5: Faster Long-Range Transformers With Conditional Co...
by junipertea on Monday March 20, 2023 @09:24PM
(#35241364)
There was no competition in academic publishing until openai made it so! The research probably started way before gpt4 became a thing.
Re:CoLT5: Faster Long-Range Transformers With Conditional Co...
by chem83 on Monday March 20, 2023 @09:56PM
(#35241603)
This may seem counter-intuitive, but model architecture is not the most significant differentiator in LLMs. Not publishing creates negative externalities around collaboration, recruiting etc. and any research organization worth their salt knows better not do avoid publishing. It's similar to the fallacy that Google should've never open-sourced Kubernetes as it allowed other cloud providers to out-compete them.
Re:CoLT5: Faster Long-Range Transformers With Conditional Co...
by xiphias2 on Tuesday March 21, 2023 @02:53AM
(#35243223)
To me model architecture sounds the most important.
Stable diffusion was possible because diffusion model architecture and then doing it in latent space instead of image space and then upscaling.
Of course someone still needed to train it, but they did it when it was theoretically possible.
Re:CoLT5: Faster Long-Range Transformers With Conditional Co...
by cma on Tuesday March 21, 2023 @03:17AM
(#35243345)
Researchers will work for less money if they are allowed to publish.
Re:CoLT5: Faster Long-Range Transformers With Conditional Co...
by Zacharias030 on Tuesday March 21, 2023 @04:29AM
(#35243738)
this.
Re:CoLT5: Faster Long-Range Transformers With Conditional Co...
by Mizza on Tuesday March 21, 2023 @06:05AM
(#35244326)
Every single decision Google makes is to avoid being seen by the government as a web advertising monopoly. Being seen as a competitor to adjacent tech companies is what they after, as they have no competitor in display advertising.