LLMs can solve equations and write code, like make a React web page.
So they assume “a story is just an equation, we just need to get the LLM to write an equation”. Wrong imo.
Storytelling is above language in the problem space. Storytelling can be done in multiple methods - visual (images), auditory (music), characters (letters and language) and gestural (using your body, such as a clown or a mime, Im thinking of Chaplin or Jacques Tati). That means storytelling exists above language, not as a function of it. If you agree with the Noah Harari vision of things, which I do, then storytelling is actually deeply connected somehow to our level of awareness and it seems to emerge with consciousness.
Which means that thinking that an LLM that can understand story because it understands language is… foolish.
Storytelling lives in a problem space above language. Thinking you can get an LLM to tell a story because you can get it to write a sentence is a misunderstanding of what problem space stories are in.
It’s the same as thinking that if an LLM can write a React web app it can invent brand new and groundbreaking computer science.
It can’t. Not for now anyway.
Do you understand why modern LLMs are different from Markov chains?
Doesn’t the AI model basically take an input of context tokens and return a list of probabilities for the next token, which is then chosen randomly, weighted by the probability? Isn’t that exactly the definition of a Markov chain?
LLMs basically return a Markov chain every single time. Think of it as a function returning a value vs returning a function.
Now, I'm sure a sufficiently large Markov chain can simulate an LLM but the exponentials involved here would make the number of atoms in the universe a small number. The mechanism that compresses this down into a manageable size is famously 'attention is a you need.!'
No, LLMs are a Markov chain. Our brain, and other mammalian brains, have feedback, strange loops, that a Markov chain doesn't. In order to reach reasoning, we need to have some loops. In that way, RNNs where much more on the right track towards achieving intelligence than the current architecture.
But no, most LLMs have a tweakae 'temperature' parameter that introduces some randomness and sometimes have very interesting results.
Yes, we can today make natural-sounding hyper-average text, images, and perhaps even music. But that's not what writing and creating is about.