it's just like Gary Marcus said

Based on my experiments it usually does get it right (18 correct answers out of 20 attempts), and the failures I got were similar to this one: a single six-letter word in an otherwise correct sentence.

Sam and friends must be giggling all the way to the bank: they have a service that 'probably' gives the correct result and paying customers are happy to retry until it gets it right.

> Sam and friends must be giggling all the way to the bank

it's true but for another reason. they yoinked it away from the nerds who were baited to work on openai because those nerds thought how the name of the company was spelled meant something about how it would behave. it reminds me of how some act around software names like 'alpha' like it has objective meaning with consequences in reality

Rumour is that there are researchers at OpenAI making 8 figure salaries. I doubt those 'nerds' are too upset about it.

"This talking dog is sort of a dumbass. I don't get the hype."

GPT is a wonder as technology goes; the hype is justified. I was discussing Sam's business model.

What have you ever bought that is always correct?

It's fed sub-word tokens not letters (even though it can split a word into letters), and apparently struggles with counting in general. No doubt some of the things it struggles with could be improved with targeted training, but others may require architectural changes.

Imagine yourself trying to use only 5 letter words if you can't see how many letters are actually in each word, and had to rely on a hodgepodge of other means to try to figure it out!

ChatGPT: You didn't say 5-non-repeat-letters, human, jez

Both the first and last words have repeating letters, so they fail under that interpretation too. There would have to be a bizarre interpretation that consecutive-repeating letters are counted as one, but non-consecutive are counted separately, for its response to be considered correct.

An AI aware of how to optimally answer questions put to it would find the least objectionable interpretation when one is a subset of the other. It also failed by not constructing a simpler sentence, like subject-verb-object or subject-verb-adjective-object, since its limitations related to letters and tokens, and its failure to double check its answers before output, mean it can make errors. The more it writes, the more chance it has of making an error.

ll is a single letter in Spanish.

ChatGPT: You didn't say I couldn't use many interpretations on the same phrase, human. ;)

Jokes apart, I think it is all about the correct prompt.