News for hackers. Stuff that matters.
Re:What Happened to Bert and T5? by vintermann on Friday July 19, 2024 @05:04PM (#41011213)
For people like me who gave up trying to follow Arxiv ML papers 3+ years ago, articles like these are gold. I would love a Youtube channel or blog which does retrospectives on "big" papers of the last decade (those that everyone paid attention to at the time) and look at where the ideas are today.
Re:What Happened to Bert and T5? by ildon on Saturday July 20, 2024 @02:17AM (#41014485)
All you need is uninterrupted attention
Re:What Happened to Bert and T5? by loa_in_ on Saturday July 20, 2024 @10:50AM (#41016864)
Maybe one needs additional heads of attention?
Re:What Happened to Bert and T5? by axpy906 on Saturday July 20, 2024 @11:18AM (#41017075)
Flash attention
Re:What Happened to Bert and T5? by swyx on Saturday July 20, 2024 @10:23AM (#41016707)
yes. my pics of youtubers (some news, but some paper reviews too) here https://github.com/swyxio/ai-notes/blob/main/Resources/Good%...