News for hackers. Stuff that matters.
Re:Keeping up with the overwhelming pace of AI innovation by timr on Tuesday March 28, 2023 @04:42PM (#35347524)
Yours is also just an opinion. But having a baseline understanding of what came before is important to having an informed opinion. To that end: GPT is not the first tech to combine "NLP" into a single model. For example, from 2019 [1].

This stuff is evolutionary.

[1] https://pub.towardsai.net/unified-language-model-pre-trainin...

Re:Keeping up with the overwhelming pace of AI innovation by og_kalu on Tuesday March 28, 2023 @07:02PM (#35349313)
You linked a paper that came months after the release of GPT-2. It is also a modified transformer, the basis of most LLMs today and an architecture that was released only two years prior.

and this is without mentioning that NLP is far from the only things LLMs do/excel at.

Frankly, nothing you have said so far has made me think your opinion is more informed than mine.

Re:Keeping up with the overwhelming pace of AI innovation by timr on Tuesday March 28, 2023 @07:46PM (#35349786)
No. I linked to a blog article which mentions many different papers, many of which were from 2018 or before. ELMO, BERT, UNILM...all examples of multi-target NLP models that pre-dated GPT.

I didn't dig this up because I thought the date on the webpage was some kind of rebuttal. I picked the first article I saw that mentioned models I knew about, with dates.

> It is also a modified transformer

Well, yes. Exactly. We're in the echo of that work, which was the core advancement that happened circa 2017. As I keep saying, it only seems revolutionary if you haven't been paying attention to the last 5 years.