I can think of no better test of whether some specific being is intelligent than fooling another human into thinking they are.

Can you think of a better one?

> because it's a behavioral or functional test

What else would you test for in an AI other than its output and behavior? Conversely: how would you test a human for intelligence other than through its output and behavior?

Surely that depends upon the human? An infant would do a terrible job. An institutionalized vegetable would fail to provide meaningful information.

A not-very-bright cousin of mine has been heard responding to robo-calls. Would his opinion do?

I think 'intelligence' is ambiguous, but many parts of it can be measured to some degree. Lets give the AI the SAT test perhaps? Or a test of hypothetical arguments. Or ask it how it would tell if you were an intelligent being yourself. Anything that's even a little bit 'meta' would confound most 'AIs'.

We can tell if an AI is functional in some environment or responds well to social queues by interacting with it. But not much else. Not how 'intelligent' it is for instance.

> Anything that's even a little bit 'meta' would confound most 'AIs'. This is precisely why the Turing Test is so tricky for machines. Knowing the parameters, any regular human could test even the most advanced present-day AI and reveal it as a machine with very little effort and only a few questions.

> We can tell if an AI is functional in some environment or responds well to social queues by interacting with it. But not much else. Not how 'intelligent' it is for instance.

The Turing Test does not intend to determine level of intelligence (it's not an IQ test). It is intended to determine whether the intelligence you're interacting with is advanced enough to fool you into thinking they are of human nature.

It is not a test of degree of intelligence, but rather of its nature. Human / not human are the only two possible outcomes.

A test that confuses a human for a machine is not a problem. False negatives don't remove value from this test.

That last bit is a false equivalence - we have no way with current science to truly introspect the "thought process" of another human, so functional judgment is the only kind we can exercise.

With AI - we built it. We can read the code that's running. We can make a distinction, in cases where output or behavior is identical, whether it's a neural network or a 12-million-line switch statement - and that value judgment means something to our determination of intelligence. There are plenty of ways other than (or in addition to) output to determine the "intelligence" of a machine that we simply can't exercise with other people.

It directly tests for intelligence, you confuse false negatives with false positives. An intelligent person that fails the test because they don't know a common language is fine.

Note: Medical tests generate both false positives and false negatives, they are still useful.

I don't understand what your example here about common language rebuts - that's something that would be solved for in the premise of the test, or it's not a fair test.

Is your argument that that person is still intelligent even if they've failed the test? Because that's the entirety of my point - the Turing test does not return a measure of intelligence, but of communicability.

Or is your point that if such a person fails it doesn't invalidate the test's measurement of intelligence? In a world where the test is implemented correctly, meaning where things like cross-language barriers are accounted for, failing to pass means failing to convince another person that you can communicate like a human would. If you fail, the only way you can consider that a "false negative" would be if you concede that the test is not a sufficient measure of intelligence but of ability to communicate - that's what makes it a FALSE negative.

No, the point is false negatives don't invalidate a test. It requires both intelligence and communication abilities.

Cost and accuracy are obvious trade-offs, but sometimes you are willing to trade say cost and false positives to avoid false negatives say, a mass screening for HIV in a blood sample. In that case you need a cheap test and while a false positive has minimal cost a false negative could be deadly.

Highering is the opposite case. Micdonalds want's a cheap test (interview + background) and as long they get enough acceptable low level candidate from their pool that's enough.

As such the turing test can fit the second example. A hypothetical AI could hold a huge range of real jobs even if limited to purely text based communications.

PS: Let's flip it. A super intelligent AI in a box that can't communicate in any form. Without IO it's indistinguishable from a space heater.

Your last example of the hyperintelligent space heater is exactly my point - the Turing test is a test of the (necessary) precondition of communication, but NOT a sufficient test of intelligence. In your example, the black box AI is assumed to be intelligent, but would still fail the Turing test - because intelligence is not a necessary condition for passing. Communicability IS necessary for the test.

A passing grade on the Turing test just indicates intelligence is possible, but excludes assured intelligence in the absence of communication ability, which is why I argue it is not a sufficient guarantee of intelligence.

You make a good point with the hiring example about the distinction between "intelligent" and "intelligent enough to do some things" - our disagreement may be stemming from us having differing definitions of "an intelligence". Need to think about it.

Nothing you just said invalidates the test.

X is a demonstration of intelligence says nothing about not X.

If you can design something that passes an arbitrary touring tests yes it is intelligent. For example I could teach it any subject that works in text format. It could then pass an open ended essay test. And get a reasonable essay.

Now, you could do the same thing with a pig and it would fail the test despite being more intelligent than the average dog. That just means there are limits.

This is one of the practical limits of the Turing Test as usually proposed - the parties could just refuse to jump through the hoop of writing an essay, and that would be a perfectly normal reaction. In practice, attempts to pass the Turing Test have leaned heavily on hand-coded strategies for changing the subject and feigning refusal to co-operate, with some success - for example https://en.wikipedia.org/wiki/Eugene_Goostman

What I find most interesting about that strategy is in practice it fails. The strength is it can push people to wait longer before judging. The weakness is it fails to demonstrate intelligence.

So help me, I think you've convinced me. I don't know what to do with all this leftover xkcd/386 though.

Thanks for the great chat.

This is also true for pwople. How do you know humans are thinking and not following lots of social rules they learn thanks to special neural hardware for learning social interaction?

Think about the intelligence conveyed in ritualized greetings.

You are recapitulating Descartes. Consciousness appears to be immeasurable. You can only know that you are conscious. It is polite to give others the benefit of doubt.

The commenter said nothing about consciousness. He simply said that just being able to hold a conversation is not a particularly useful measure of intelligence. The broader form of that is that we don't even know if humans in general are intelligent.

You can know that you are conscious, but how do you know you are intelligent?