I do think some known weaknesses do make it problematic though; some intelligent behaviour might be not human; aka maybe the player is a generally super intelligent computer but it’s answers are so out there the other players do not recognise it as intelligence. Or the human player is so weird, the answers are mistaken for a computer. Or, the most likely and I have played this many times, lately with gpt3; human behaviour often is not intelligent, at all. And while computers are not intelligent, in a scenario against someone who just responds with emoji’s and such (so, a currently normal convo on the internet), the interrogator might appoint humanness to the wrong side many or all times. As we already know that people who do not know they are chatting to a bot, think it is a human.

While I do not think that is intelligence, it does making testing for it slightly broken. It will come down to ‘I will know it when I see it’ by elitists for exactly the above.

So in short ; I do think the Chinese room argument against it is quite a good one where even the elitist people can fall for if the ‘con’ is elaborate enough (as in; if gpt gets more data, more efficient learning and learns to know when it cannot answer and has to look it up, like calculations). Or maybe then it is intelligent? (To be clear; I don’t think so; I think we need a better tests and definitions).

Which one? Turing himself revised it, like, four times. The version where humans can't tell which one chat member is non-human has been beaten several times (with tricks like mimicking a 13 year old foreign boy, iirc). The version where you replace an arbitrary member of a chat with a human is something mostly about speech pattern matching and plausible speech, can't imagine that GTP3 or whatever comes next is too bad at that. If you do it like in Ex Machina, as in chatting in person it's mostly about building a very expensive robot that overcomes uncanny valley.

I think the sad state is that the Turing Test is too diluted to be useful as a marker...

I expect "passing the Turing test" not to be a binary thing, but as AI improves it can hold longer and more nuanced discussions before the human can detect that it's a machine. So I'm not surprised that we already have chatbots that can fool some people some of the time.

Yes, but that makes it a pretty elitist thing no? Maybe elitist is the wrong word (not a native english speaker), but I mean; I would be able to always make the test fail for the computer if I am the interrogator because because. I cannot say why. Something like you say and something where I think that there is a very large % of the world population who will not be able to make that distinction.

An interesting would be to do the Turing test with a timer; the interrogator gets 30s for each session and has to say who is what within 30s and then continue to the next batch. I think computers would go very very far if you do that. But that is the attention span consider normal currently for human/human social media interaction; often even less.

You need to distinguish humans from computers, you don't know whether you're talking to a computer or not. Failing everybody doesn't work. Maybe I'm not getting what you're trying to do.