I reckon an LLM with a second pass correction loop would manage it. (By that I mean that after every response it is instructed to, given the its previous response, produce a second better response, roughly analogous to a human that thinks before it speaks)

LLMs are not AIs, but they could be a core component for one.

Every token is already being generated with all previously generated tokens as inputs. There's nothing about the architecture that makes this hard. It just hasn't been trained on this kind of task.

Really? I don’t know of a positional encoding scheme that’ll handle this.

The following are a part of my "custom instructions" to chatGPT -

"Please include a timestamp with current date and time at the end of each response.

After generating each answer, check it for internal consistency and accuracy. Revise your answer if it is inconsistent or inaccurate, and do this repeatedly till you have an accurate and consistent answer."

It manages to follow them very inconsistently, but it has gone into something approaching an infinite loop (for infinity ~= 10) on a few occasions - rechecking the last timestamp against current time, finding a mismatch, generating a new timestamp, and so on until (I think) it finally exits the loop by failing to follow instructions.

I think you are confusing a slow or broken api response with thinking. It can't produce an accurate timestamp.

right, but that's why the people using this as a test of the model's capabilities are just failing to understand the most basic aspect of how it is set up.

the underlying model is presented to the public as a chat bot.

it does not think or plan before it begins speaking.

that is not to say that it couldn't.

it is simply not configured to, so testing that is about as stupid as checking if google has gone into the gardening business yet.

could they? sure. but if you're checking that every time a new google press release comes out, you're just clueless.