It is absolutely impressive. It's also absolutely not reasoning.
Re:Input to GPT Matters a Lot
by BoardsOfCanada on Monday April 03, 2023 @11:30AM
(#35425428)
It impressively emulates the output of reasoning without reasoning.
Indeed. It will prove to be a useful tool as its capabilities are mapped to workflows.
My argument is nothing against it as a technical achievement, and everything against the sophistry that it is a reasoning engine.
It is very impressive. However, the fact that it manages a buffer of something like 30k tokens (or whatever new limit it is) means that it's definitely autocomplete and not something with reasoning. There will be a lot of things it can't do because of that limit. That limit will also increase the amount of forgetfulness GPT4 applications have because companies using it will "auto-magically" work around the limit by sending different chunks of history to it to get whatever from it. That also means that GPT4 applications will generally be architected to have that limit - which means weirdness - non-coherent completions or code generations that "forget" something important.
This is very different from real reasoning that has unlimited (yet imperfect) memory.
What the heck is real reasoning with unlimited memory? Human minds are quite limited! Our short term context length is certainly not 30k words. Most psychological research says it’s about 7 items.
Wow, this is conflating things that are not the same in an incredibly misleading way.
Hey, that was 15 words, and I had them all in my head at once before I started typing! I was able to form a sentence of more than seven words all by myself!
You seem to be misapplying the concept of working memory, where seven (plus or minus two) is about arbitrary and unrelated words or numbers, not "items." In fact, it's generally seven for numbers, but lower for words. And again, that's unrelated words, like "person woman man camera tv," but probably even more unrelated.
Did you really have those 15 words in your head all at once or did you just predict 15 next words that would form a good reply to my words?
One of the main lessons this whole GPT fiasco has taught us is that if you make a super-duper fancy autocomplete trained on terabytes of textual data, it develops some sort of internal logic/understanding. That is very exciting to me.
It is truly impressive.