News for hackers. Stuff that matters.
Re:AI is becoming 'yes-men on servers' by mentalgear on Monday March 10, 2025 @05:32AM (#43318493)
BlueSky version: https://bsky.app/profile/thomwolf.bsky.social/post/3ljpkl6c6...

---

Quite interesting post that asks the right question about "asking the right questions". Yet one aspect I felt missing (which might automatically solve this) is first-principles-based causal reasoning.

A truly intelligent system — one that reasons from first principles by running its own simulations and physical experiments — would notice if something doesn't align with the "textbook version".

It would recognize when reality deviates from expectations and ask follow-up questions, naturally leading to deeper insights and the right questions - and answers.

Fascinating in this space is the new "Reasoning-Prior" approach (MIT Lab & Harvard), which trains reasoning capabilities learned from the physical world as a foundation for new models (before evening learning about text).

Relevant paper: "General Reasoning Requires Learning to Reason from the Get-go."

Re:AI is becoming 'yes-men on servers' by mentalgear on Monday March 10, 2025 @05:38AM (#43318532)
Re:AI is becoming 'yes-men on servers' by zombot on Tuesday March 11, 2025 @02:56AM (#43329859)
"Reality" is an alien concept to an LLM. All they have is sequences of words that they can complete.
Re:AI is becoming 'yes-men on servers' by reverius42 on Tuesday March 11, 2025 @06:16AM (#43330960)
"Reality" is an alien concept to a Human. All they have is sequences of motions and noises they can complete.
Re:AI is becoming 'yes-men on servers' by neom on Tuesday March 11, 2025 @09:44AM (#43332379)
I explained to someone mum is mum because of lips, air and sounds and they didn't believe me, so that's what we're contending with....
Re:AI is becoming 'yes-men on servers' by herculity275 on Tuesday March 11, 2025 @09:11AM (#43332073)
I used to think that but newer models' increasing coherence coupled with the astonishing success of RLHF makes me think that LLMs that can effectively intuit "reality" will emerge fairly soon.
Re:AI is becoming 'yes-men on servers' by alex77456 on Wednesday March 12, 2025 @01:02AM (#43340100)
Maybe we're overestimating average human brain capabilities
Re:AI is becoming 'yes-men on servers' by chezelenkoooo on Wednesday March 12, 2025 @02:21AM (#43340421)
I think the evidence of all human progress disagrees with this statement fairly strongly.
Re:AI is becoming 'yes-men on servers' by alex77456 on Wednesday March 12, 2025 @03:16AM (#43340638)
Human progress is driven by a small percentage of the population
Re:AI is becoming 'yes-men on servers' by sage92 on Wednesday March 12, 2025 @11:26AM (#43344257)
The human body is driven by a small percentage of the overall genome. It remains to be seen if that small percentage really doesn’t play a part… we tend to remember those who scored the goal, but often forget about what it took for the scorer to have a shot in the first place…
Re:AI is becoming 'yes-men on servers' by antonkar on Tuesday March 11, 2025 @10:42AM (#43332929)
Interesting, I think the guy who’ll make the GUI for LLMs is the next Jobs/Gates/Musk and Nobel Prize Winner (I think it’ll solve alignment by having millions of eyes on the internals of LLMs), because computers became popular only after the OS with a GUI appeared. I just started ASK HN to let people share their AI safety ideas, both crazy and not: https://news.ycombinator.com/item?id=43332593