News for hackers. Stuff that matters.
Re:Did GoogleAI just snooker one of Silicon Valley’s sharpes... by darawk on Thursday September 15, 2022 @08:17PM (#32860287)
It's a lame test, but I don't think most people were claiming that it proves general compositionality. What it does prove is that compositionality is possible with these models, and will likely improve rapidly, as everything else has that they've gotten a toehold into.

Ironically, the very fact that there is now a compositionality benchmark, as Gary points out, is all you really need to know that it's going to fall in the next decade, and probably sooner than that. I'm not aware of any major benchmark dataset upon which enormous progress has not been made in the last few years. And i'd be more than willing to bet anyone anything they'd like that a great deal of progress will be made on this one over the next few.

Re:Did GoogleAI just snooker one of Silicon Valley’s sharpes... by abrax3141 on Thursday September 15, 2022 @08:25PM (#32860348)
Marcus seems to be treating it as the hallmark of intelligence (I think he actually uses that phrase), so arguing about whether the hack manages to get the tree into the effective object slot vs the effective subject slot is really not much of a hallmark.