News for hackers. Stuff that matters.
Re:Meet the $4 Billion AI Superstars That Google Lost by riku_iki on Saturday August 19, 2023 @01:04PM (#37190805)
invention of transformer was not pivotal, those t2t guys who authored paper couldn't achieve any quality improvements over other SOTAs at that time.

Pivotal moment was BERT, which demonstrated that with some additions transformer can achieve very significant improvements, and inventor is still at google, and transformer creators got lot of hype from that.

Also, most important part is attention, which also was not invented at google.

Re:Meet the $4 Billion AI Superstars That Google Lost by visarga on Sunday August 20, 2023 @01:36PM (#37201253)
This is an "original" take to put BERT above Attention is All You Need.
Re:Meet the $4 Billion AI Superstars That Google Lost by riku_iki on Sunday August 20, 2023 @01:50PM (#37201384)
what do you mean? could you explain?