Re:Geoffrey Hinton leaves Google and can now speak freely ab...
by bitL on Monday May 01, 2023 @11:41AM
(#35774399)
GPT basically showed that scalable brute-force trumps clever theoretical models which makes many academics salty.
[ Parent
| on HN ]
Re:Geoffrey Hinton leaves Google and can now speak freely ab...
by mrazomor on Monday May 01, 2023 @01:15PM
(#35775954)
That's something that Microsoft research wrote two decades ago. And those results were well known in the NLP community.
Example: https://www.microsoft.com/en-us/research/wp-content/uploads/... (Michele Banko published a few similar papers on that topic)
Re:Geoffrey Hinton leaves Google and can now speak freely ab...
by bitL on Monday May 01, 2023 @07:02PM
(#35780503)
There was no hugely scalable approach before transformers, RNNs, the previous SOTA, were notoriously bad at scaling.
Re:Geoffrey Hinton leaves Google and can now speak freely ab...
by tensor on Monday May 01, 2023 @08:20PM
(#35781338)
Yes, we needed clever ideas from scientists to make them scale. In fact, we still need clever ideas to make them scale because the current architectures still have all sorts of problems with length and efficiency.