Re:ChatGPT: It has no inner monologue or meta-cognition
by monk_e_boy on Tuesday May 02, 2023 @06:27AM
(#35785507)
When ChatGPT first came out I was asking it to draw pictures of goblins and wizards using Python and Turtle - it's pretty bloody good at it. You can ask it to make the goblin angry or happy or whatever.
There is some sort of magic in which the LLM is imagining what a goblin looks like, then converts those thoughts to turtle commands and then finally adds python. It's quite impressive.
For example, the eyes, head, hair, horns etc are all in the correct place.... how? Using imagination?
Re:ChatGPT: It has no inner monologue or meta-cognition
by ojosilva on Tuesday May 02, 2023 @06:58AM
(#35785718)
I think somewhere down the rabbit hole we lost perspective of what neural nets are doing and how we got to the post-GPT world. Remember deep neural-networks? The idea is that each layer _deeper_ the software encodes more _hidden_ relationships from the training data. That's where this "magic" is embedded. At certain depths the neural-net stores relationships that go beyond our human reasoning and conscious understanding, and the software results generated from these depths are unsettling and tend to startle us. Interesting enough, these are things we are perfectly capable of doing but unable to actually query in our heads as we, just like ChatGPT, do not posses the ability to physically examine our own neural-net content.
A simple example is to ask it to write a complex program in a language like Java or Kotlin that requires you to declare all your imports up front. There can be a lot of these. Therefore the model must figure out what it wants to use very far down the program in order to emit the first words of the program correctly. Presumably somewhere in the intermediate state process the network is actually working out the entire program ahead of time so it can predict the next word of it, and that's sufficiently stable for it to do so successfully.
Also I'm not quite sure I understand what you mean by not maintaining hidden state between tokens. Doesn't the transformer algorithm start by parallel processing every token of the prompt to compute a large set of vectors and matrices that are then reused over and over whilst generating the results? What are the KV vectors if not hidden state of some sort?