Re:Entropy of a Large Language Model output
by Vetch on Monday January 13, 2025 @02:55PM
(#42688155)
This seems wrong. Reasoning scales all the way up to the discovery of quaternions and general relativity, often requiring divergent thinking. Reasoning has a core aspect of maintaining uncertainty for better exploration and being able to tell when it's time to revisit the drawing board and start over from scratch. Being overconfident to the point of over-constraining possibility space will harm exploration, only working effectively for "reasoning" problems where the answer is already known or nearly fully known. A process which results in limited diversity will not cover the full range of problems to which reasoning can be applied. In other words, your statement is roughly equivalent to saying o3 cannot reason in domains involving innovative or untested approaches.
Re:Entropy of a Large Language Model output
by larodi on Monday January 13, 2025 @05:18PM
(#42690225)
> Reasoning scales all the way up to the discovery of quaternions and general relativity, often
That would be true only if all that we grant for based/true/fact came through reasoning in a complete logical and awoke state. But it did not, and if you dig a little or more you'd find a lot of actual dreaming revelation, divine and all sorts of subconscious revelation that governs lives and also science.
Re:Entropy of a Large Language Model output
by BobbyJo on Monday January 13, 2025 @11:11PM
(#42693519)
I'd also like to point out serendipitous external input as well. Isaac Newton and watching the apple fall from the tree for instance. Often, thought processes are steered by external stimuli that happen to occur while the thought process is taking place.
Re:Entropy of a Large Language Model output
by larodi on Wednesday January 22, 2025 @05:13PM
(#42798075)
its steered by magic or magick at best, and too many have tried to devise the principle behind it. math in all its beauty is too simple to grasp it yet if ever. noone does, that's the beauty of life. and nobody wants an ugly life.
Of course humans employ different thinking modes too - no harm in thinking like a stone cold programmer when you are programming, as long as you don't do it all the time.