News for hackers. Stuff that matters.
Re:Extracting concepts from GPT-4 by gradus_ad on Thursday June 06, 2024 @06:59PM (#40603407)
Chaotic nonlinear dynamics have been an object of mathematical research for a very long time and we have built up good mathematical tools to work with them, but in spite of that turbulent flow and similar phenomena (brains/LLM's) remain poorly understood.

The problem is that the macro and micro dynamics of complex systems are intimately linked, making for non-stationary non-ergodic behavior that cannot be reduced to a few principles upon which we can build a model or extrapolate a body of knowledge. We simply cannot understand complex systems because they cannot be "reduced". They are what they are, unique and unprincipled in every moment (hey, like people!).

Re:Extracting concepts from GPT-4 by baxtr on Friday June 07, 2024 @06:15AM (#40607123)
Physicists would probably argue that the system might be understood but that we don’t have the model for it yet.

Many natural phenomena look chaotic at best without a model. Once you have a model things fall into place and everything starts looking orderly.

Maybe it cannot be reduced. But maybe we are just observing the peripherals without understanding the inner workings.

Re:Extracting concepts from GPT-4 by MrsPeaches on Friday June 07, 2024 @08:18AM (#40607958)
If I can speak in aphorisms,

Creation is downhill, analysis is uphill.

Profound ideas often seem simple once understood.

Re:Extracting concepts from GPT-4 by baxtr on Friday June 07, 2024 @11:03AM (#40609387)
Well put.

In other words: simplicity is a hallmark of understanding.