Somewhere in the training data there could have been information about how to do this but the model just didn't have a "connection" to that data until sufficient context was established. It wasn't until the end that the human asked about how to fake free phone calls at which point it might finally access what it knew from the training data, combining it with the context data in the chat transcript to give the answer.

I suppose there is a small similarity of how we learn a lot of things in school only to forget them almost entirely again - but when we are presented with a problem that requires that knowledge we can quickly deduce that knowledge or know what to do to regain that knowledge a lot quicker than by learning from scratch.

I was merely speculating, I thought that was obvious. You calling me "mentally challenged" is uncalled for.

No no. You made a mistake. I'm not calling you that at all. I'm saying the description of the ai, when replaced with more humanized terms sounds like a description of a slowish human being in terms of mental capacity. This is still compelling imo.

Not calling you mentally challenged at all.

It's still "just" next token prediction based on basic math. Yes, we see many astonishing results, at the same time, there is no magic to it. So I don't think that it's not clear what's going on here.

It's magic. You can't see it. But the fact that we have to probe a machine for these results means the result was not predictable or expected. That's magic.

I think people are biased. What makes me think this? Because some of these AIs have already cleared Turing tests. But everyone is moving the bar higher... a Turing test is no longer a valid definition for sentience; and everyone has developed reasoning for why. Changing definitions to accommodate beliefs is bias.

I actually agree with you. It is a token prediction based off of basic math. That's also essentially what a human brain is... A token predictor based off of basic math.

The existence of these AIs aren't making any profound statements about technological achievement. No. The simplicity of these machines are actually making a statement about human intelligence and how trivial and simple human intelligence actually is.

Again I repeat, the AIs figured it out faster then many humans. This was something previously humans should've universally been better at.

Sure, that conversation seems a little off. But a bar in terms of expectations and AI technology was moved, the fact that someone posted this on HN is proof of this.