But saying GPT-2 to 4 in 4 years isn't revolutionary just seems contrarian to me.
Bespoke NLP of any kind is out the door with a single model. That's fucking crazy.
This stuff is evolutionary.
[1] https://pub.towardsai.net/unified-language-model-pre-trainin...
and this is without mentioning that NLP is far from the only things LLMs do/excel at.
Frankly, nothing you have said so far has made me think your opinion is more informed than mine.
I didn't dig this up because I thought the date on the webpage was some kind of rebuttal. I picked the first article I saw that mentioned models I knew about, with dates.
> It is also a modified transformer
Well, yes. Exactly. We're in the echo of that work, which was the core advancement that happened circa 2017. As I keep saying, it only seems revolutionary if you haven't been paying attention to the last 5 years.