News for hackers. Stuff that matters.
Re:Coconut by Meta AI – Better LLM Reasoning with Chain of C... by konart on Tuesday December 31, 2024 @10:37AM (#42559308)
Not magically. Our great ancestors were pretty dumb, but they were getting smarter and better because of sharing their knowledge.
Re:Coconut by Meta AI – Better LLM Reasoning with Chain of C... by pigpop on Tuesday December 31, 2024 @11:26AM (#42559686)
yes they got "smarter" by compiling a corpus of knowledge which future generations could train on.

sarcasm aside, throwing away the existing corpus in favor of creating a new one from scratch seems misguided.

this paper isn't about creating a new language, they are omitting the sampler that chooses a single token in favor of sending the entire end state back in to the model like a superposition of tokens. that's the breadth first search part, they don't collapse the choice down to a single token before continuing so it effectively operates on all of the possible tokens each step until it decides it's done.

it would be interesting to try this with similar models that had slightly different post training if you could devise a good way to choose the best answer or combine the outputs effectively or feed the output of a downstream model back in to the initial model, etc. but I'm not sure if there'd necessarily be any benefit to this over using a single specialized model.

Re:Coconut by Meta AI – Better LLM Reasoning with Chain of C... by ulbu on Tuesday December 31, 2024 @10:59AM (#42559493)
they were not one bit dumber than you.
Re:Coconut by Meta AI – Better LLM Reasoning with Chain of C... by EliBullockPapa on Tuesday December 31, 2024 @05:44PM (#42562407)
Average intelligence measures have risen substantially since early 1900s

https://en.wikipedia.org/wiki/Flynn_effect