Re:GPTs Are GPTs: An Early Look at the Labor Market Impact P...
by pwinnski on Monday March 20, 2023 @11:33AM
(#35233085)
I had an extended back-and-forth in which I asked ChatGPT how to undo mistakenly marking a message as junk in Messages on iOS.
It gave me a 7-step answer that was completely wrong. Then I said no, you can't do that, it apologized and gave me an 8-step answer that was completely wrong. When I pointed out in this case there is no "Junk" folder in iOS Messages, it apologized for the confusion and gave me another 8-step answer that was completely wrong. When I pointed out why that one wouldn't work, it gave up and said recovery was impossible and that I would need to contact the sender to re-send, and be more careful when marking messages as junk. This was still wrong, as recovery is possible, just not by any of the means it described.
So yeah, I have been super-impressed by the quality of output from these LLMs, but I cannot imagine actually relying on one for anything where correctness matters.
Giving me a nice list of Korean shoegaze bands, sure. Its step-by-step for how to become a better volleyball player will be great for my daughter, and was better than the answer to the same question from Google. But correctness? No.
It gave me a 7-step answer that was completely wrong. Then I said no, you can't do that, it apologized and gave me an 8-step answer that was completely wrong. When I pointed out in this case there is no "Junk" folder in iOS Messages, it apologized for the confusion and gave me another 8-step answer that was completely wrong. When I pointed out why that one wouldn't work, it gave up and said recovery was impossible and that I would need to contact the sender to re-send, and be more careful when marking messages as junk. This was still wrong, as recovery is possible, just not by any of the means it described.
So yeah, I have been super-impressed by the quality of output from these LLMs, but I cannot imagine actually relying on one for anything where correctness matters.
Giving me a nice list of Korean shoegaze bands, sure. Its step-by-step for how to become a better volleyball player will be great for my daughter, and was better than the answer to the same question from Google. But correctness? No.