News for hackers. Stuff that matters.
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by thomasahle on Monday May 01, 2023 @11:12AM (#35773927)
My experience in "Applied Research" is that often "good ideas from lower ranked researchers" (or good ideas from anyone really) is "I saw this cool paper, let's try and implement that". That doesn't mean top people like Hinton should get all the credit, but let's not kid ourselves and believe most of the ideas didn't origin in academia.

One of GOpenAI's recent breakthroughs was switching to FlashAttention, invented at Stanford and University at Buffalo.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by lostmsu on Monday May 01, 2023 @04:00PM (#35778325)
Isn't Hinton's paper on backprop just a "let's try to implement that" for a multilayer network?
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by thomasahle on Monday May 01, 2023 @05:18PM (#35779312)
I'm not going to claim Hinton "invented backprop". He even says he didn't himself: https://www.reddit.com/r/MachineLearning/comments/g5ali0/com...