Re:GPT-4 could pass Bar Exam, AI researchers say
by Aperocky on Tuesday January 03, 2023 @07:09PM
(#34239992)
Language prediction model is not a closely guarded secret, I suggest looking into academic papers about what they are and maybe even see/do some implementation yourself.
There are no magic, it is just a more complicated transpose, created by training over perhaps 10% of all available text on the internet.
It does have a lot of use, for one I think it would probably put grammarly out of business, and maybe even do some work for law firms.
I also really strongly disagree that it’s basically doing some sort of information retrieval design where based on language it regurgitates some sort of markov expectations. You can ask it to do very complex translations of a concept from one domain to another and expressed in a form that’s certainly never been done before and it does it with alacrity. At the very minimum it “remembers” things from the past in the conversation and can associate the semantic ideas across prompts and synthesize cogent responses - that in itself implies it has some semantic “understanding” of the structure of the language. That is a huge missing piece in our tool kit to date.
Frankly I feel these threads expose just how jaded and unable to dream we have become, that even when a wonder walks up and hits you in the nose we can’t even see it.