News for hackers. Stuff that matters.
Re:Transformer Attention is off by one by jshen on Monday July 24, 2023 @07:30PM (#36855652)
I've read a lot of scientific papers in the comp sci / machine learning space and they are rarely precise. It's been over a decade since I've ready many papers so maybe this has changed, but I remember reading a paper out of Microsoft about how to make spell correcting auto-completion for search, and it was nearly impossible to figure out precisely how it was implemented. Precision would have been achieved easily by providing code and a sample data set. instead of was a mix of prose and math equations with many gaps where you had to guess how to fill.
Re:Transformer Attention is off by one by a_bonobo on Monday July 24, 2023 @08:14PM (#36856029)
Ah yes, my old supervisor was very fond of that strategy.

"Make it sound like we do cool stuff; but don't make it so precise that they can re-implement what we do. Let them come to us so we can co-author papers."