News for hackers. Stuff that matters.
Re:The Collapse of GPT by mondrian on Friday May 16, 2025 @08:35PM (#44011081)
The "core reasoning" part of AI may be increasingly important to improve, and its "database of factual knowledge" aspects may be less and less important, maybe increasingly a hindrance. So more focused and specialized training may take over toward increasing reasoning precision, and not this never-ending stream of new data.

So maybe we'll get better reasoning and therefore better generated data/content in the wild, without this negative feedback loop everyone is worried about.

Re:The Collapse of GPT by meroes on Friday May 16, 2025 @08:44PM (#44011131)
What core reasoning?
Re:The Collapse of GPT by mondrian on Friday May 16, 2025 @11:52PM (#44011915)
Re:The Collapse of GPT by meroes on Saturday May 17, 2025 @06:48PM (#44017594)
That’s amazing because made up language might also just be context scaffolding sans reasoning, e.g. it’s arbitrary extra context for machines to relate human text better. I’m not even trying to play devils advocate—-like both sides, true believers or pessimists, come up with wholly unconvincing arguments. (I genuinely don’t know if the tweet is a true believer or not). At least the pessimists aren’t coupled with the AI marketeers.
Re:The Collapse of GPT by mondrian on Sunday May 18, 2025 @04:15AM (#44019808)
Also in the vicinity: https://www.anthropic.com/research/tracing-thoughts-language...

There's also distillation, where you can drastically improve a small model by training it on chains of thoughts of larger models. You can't achieve the same performance by training on original human texts. This suggests that those chains of thoughts reliably contain "densely packed reasoning", meaning the LLM probably has developed internal clusters of "reasoning circuitry", loosely speaking.

Re:The Collapse of GPT by SpicyLemonZest on Friday May 16, 2025 @09:22PM (#44011318)
For example, I just asked o3 "how many Is are in antidisestablishmentarianism", and this was its train-of-thought output:

------

The user wants to know how many “I”s are in the word “antidisestablishmentarianism.” They probably mean the letter "I" in lowercase, though the question was a little broader with "Is."

I’ll treat it case-insensitively and count the total occurrences of the letter "i" (both uppercase and lowercase). I can count them manually now, or use a tool for faster analysis.

Analyzed word = "antidisestablishmentarianism" word_lower = word.lower() word_lower.count('i')

5 The user asked how many "I"s are in "antidisestablishmentarianism." I’m taking it to mean whether it’s counting "i" or "I", and I’ll consider them as the same for simplicity. So, I’ll count the 5 occurrences of "i" in the word. I manually broke it down and verified there are indeed five "i"s, so the final count should be 5. Pretty straightforward, now that it's all clear!

Done

There are 5 “i”s in “antidisestablishmentarianism.”

------

You can come up with complicated definitions of "core reasoning" where this doesn't count, and I understand there's great demand for such definitions, but this clearly satisfies common-sense understanding of the term.

Re:The Collapse of GPT by Jensson on Friday May 16, 2025 @11:00PM (#44011716)
> You can come up with complicated definitions of "core reasoning" where this doesn't count

Did we read the same response? It did write a lot of reasons, but didn't do any reasoning at all, it just suddenly wrote "5" here

    So, I’ll count the 5 occurrences of "i" in the word.
There was no reasoning at all to arrive at 5, so no your example just proves how these models are great at faking reasoning.
Re:The Collapse of GPT by snmx999 on Saturday May 17, 2025 @02:51AM (#44012507)
What kind of response would have satisfied you?
Re:The Collapse of GPT by selfhoster on Friday May 16, 2025 @09:49PM (#44011439)
Then I guess Ubuntu has had reasoning for several decades:

    sudp
    Command 'sudp' not found, did you mean:
      command 'sudo' from deb sudo (1.9.9-1ubuntu2.4)
      command 'sudo' from deb sudo-ldap (1.9.9-1ubuntu2.4)
      command 'sup' from deb sup (20100519-3)
      command 'sfdp' from deb graphviz (2.42.2-6)
    Try: sudo apt install <deb name>
Re:The Collapse of GPT by meroes on Friday May 16, 2025 @09:58PM (#44011477)
I might just be on the opposite side of the aisle, but to me chain-of-thought is better understood as simply more context.

Of course there is ambiguity though, more context would be hard to distinguish from core-reasoning and vice versa.

I think LLMs/AI mean we can substitute reasoning with vast accumulations and relations between contexts.

Remember, RLHF gives the models some, and perhaps most of these chains-of-thought, when there isn’t sufficient text to scrape for each family of problems. When I see that chain-of-thought, the first thing I think of is of my peers who had write, rewrite, nudge, and correct these chains of thought, and not about core reasoning.

The CoT has that same overexplained step-by-step so many RLHF’ers will be accustomed to, and much of it was authored/originated by them. And due to the infinite holes it feels like plugging, I dont call that RL reasoning.

Re:The Collapse of GPT by gerdesj on Friday May 16, 2025 @08:39PM (#44011103)
You seem to be arguing that bollocks couched in flowery language is a stretch goal of AI.

Are you sure?

Re:The Collapse of GPT by _heimdall on Friday May 16, 2025 @08:51PM (#44011170)
That really depends on expectations.

If AI is meant to sound nearly identical to a human, you don't need more training data.

If its meant to act as a natural language encyclopedia, we'll never get there with LLMs which amount to natural language processing on top of a massively compressed dataset.

Re:The Collapse of GPT by gerdesj on Friday May 16, 2025 @09:13PM (#44011269)
I've read the Google page "Getting AI to write good SQL: Text-to-SQL techniques explained" and it looks like all the usual bollocks.

... and sigh ... it look like I'm twiddling with a noddy!

Re:The Collapse of GPT by mondrian on Sunday May 18, 2025 @04:21AM (#44019834)
Trying to make AIs more factually-accurate with more training is probably hopeless. Current events and encyclopedic knowledge will be provided by tools. The LLM's core job is to choose the right tools for the job and synthesize their outputs.