LLMs are not greater than the sum of their parts: Stanford researchers

Re:LLMs are not greater than the sum of their parts: Stanfor... by ljlolel on Monday May 08, 2023 @08:15PM (#35868681)

This response is very poor and ignores many very well-developed arguments in the Stanford paper (such as incorrect NLP regex, or that exact answer nonlinearity can still be measured more finely with larger number of exact answer test questions)

[ Parent | on HN ]

Re:LLMs are not greater than the sum of their parts: Stanfor... by jiocrag on Monday May 08, 2023 @08:18PM (#35868707)

It’s literally addressed in his first bullet point…

“ Response: While there is evidence that some tasks that appear emergent under exact match have smoothly improving performance under another metric, I don’t think this rebuts the significance of emergence, since metrics like exact match are what we ultimately want to optimize for many tasks.”

[ Parent | on HN ]

Re:LLMs are not greater than the sum of their parts: Stanfor... by ljlolel on Monday May 08, 2023 @08:22PM (#35868735)

You haven’t read the paper so you don’t understand what I just wrote and why Wei is not responding to the paper. It’s not a different metric

[ Parent | on HN ]

Re:LLMs are not greater than the sum of their parts: Stanfor... by jiocrag on Tuesday May 09, 2023 @08:25PM (#35882111)

I did read the paper and Wei is responding to it, and others. What do you mean "It's not a different metric?"

[ Parent | on HN ]