Conventional LLM chatbots behave the way you describe because their goal during training is to as much as possible impersonate an intelligent assistant.

Do you think this goal during training cannot be changed to impersonate someone normal such that you cannot detect you are chatting with an LLM?

Before flight was understood some thought "magic" was involved. Do you think minds operate using "magic"? Are minds not machines? Their operation can not be duplicated?

I'm not the person you asked, but I think:

1. Minds are machines and can (in principle) have their operation duplicated

2. LLMs are not doing this

> Do you think this goal during training cannot be changed to impersonate someone normal such that you cannot detect you are chatting with an LLM?

I don't think so, because LLMs hallucinate by design, which will always produce oddities.

> Before flight was understood some thought "magic" was involved. Do you think minds operate using "magic"? Are minds not machines? Their operation can not be duplicated?

Might involve something we don't grasp, but despite that: only because something moves through air it's not flying and will never be, just like a thrown stone.

Maybe current LLMs can do that. But none are, so it hasn't passed. Whether that's because of economic or marketing reasons as opposed to technical does not matter. You still have to pass the test before we can definitely say you've passed the test.

Overall I'd say the easiest is just overall that the models always just follow what you say and transform it into a response. They won't have personal opinions or experiences or anything, although they can fake it. it's all just a median expected response to whatever you say.

And the "agreeability" is not a hallucination, it's simply the path of least resistance, as in, the model can just take information that you said and use that to make a response, not to actually "think" and consider I'd what you even made sense or I'd it's weird or etc.

They almost never say "what do you mean?" to try to seek truth.

This is why I don't understand why some here claim that AGI being already here is some kind of coherent argument. I guess redefining AGI is how we'll reach it

I agree with your points in general but also, when I plugged in the parent comment's nonsense question, both Claude 4.5 Sonnet and GPT-5 asked me what I meant, and pointed out that it made no sense but might be some kind of metaphor, poem, or dream.

What did you plug in?

If it wasn't structured as a coherent conversation, it will ask because it seems off, especially if you're early in the context window where I'm sure they've RLd it to push back, at least in the past year or so

And if it's going against common knowledge or etc which is prevalent in the training data, it will also push back which makes sense