News for hackers. Stuff that matters.
Re:Chain-of-Thought Can Hurt Performance on Tasks Where Thin... by Jensson on Wednesday October 30, 2024 @11:03PM (#42003068)
> As I'm writing this, I'm deciding the next few words to type based on my last few.

If so you could have written this as a newborn baby, you are determining these words based on a lifetime of experience. LLMs doesn't do that, every instance of ChatGPT is the same newborn baby while a thousand clones of you could all be vastly different.

Re:Chain-of-Thought Can Hurt Performance on Tasks Where Thin... by thomashop on Thursday October 31, 2024 @12:06AM (#42003403)

  We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time and across multiple domains, the ways by which different neural networks represent data are becoming more aligned. Next, we demonstrate convergence across data modalities: as vision models and language models get larger, they measure distance between datapoints in a more and more alike way. We hypothesize that this convergence is driving toward a shared statistical model of reality, akin to Plato’s concept of an ideal reality. We term such a representation the platonic representation and discuss several possible selective pressures toward it. Finally, we discuss the implications of these trends, their limitations, and counterexamples to our analysis.
https://arxiv.org/html/2405.07987v5