Newsgroups: comp.ai.philosophy
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!swrinde!sgiblab!wetware!spunky.RedBrick.COM!psinntp!psinntp!psinntp!scylla!daryl
From: daryl@oracorp.com (Daryl McCullough)
Subject: Re: Bag the Turing Test
Message-ID: <1994Dec8.143144.11787@oracorp.com>
Organization: Odyssey Research Associates, Inc.
Date: Thu, 8 Dec 1994 14:31:44 GMT
Lines: 46

Several people have claimed that passing the Turing Test is too
*difficult*---that there could very well be a being that is as
intelligent as a human, but whose mind is so different from ours that
it couldn't pass a Turing Test. I think that is a good point when it
comes to testing the intelligence of alien races, but I don't really
think it matters much for Artificial Intelligence. If an AI program
were actually as intelligent as a human then I think it could easily
be modified to pass the Turing Test. (It's not so easy to reprogram
a living being, though.)

So, in spite of the title of this thread, I have to say that I
consider the Turing Test just about perfect as a test for artificial
intelligence: (1) I think that any sufficiently intelligent program
could be modified to pass the Turing Test, and (2) I believe that it
is impossible to pass a stringent enough Turing Test (*) without real
intelligence.

Here is my candidate for a Turing Test for AI programs which would be
practically impossible to fool. You need three participants: the
judge, the AI's Advocate and the Devil's Advocate. The goal of the
Devil's advocate is to convince the judge that the subject is merely
an unintelligent computer program, and the goal of the AI's Advocate
is to convince the judge that the subject possesses genuine
intelligence.  However, the only means that the two advocates have of
convincing the judge is by asking the subject questions and letting
the judge hear the responses. The advocates are able to analyze the
code (to find out *how* responses are created) in order to come up
with their questions. But the judge is not allowed to see the code,
and is not even allowed to know whether the subject is human or
machine.

I think it is clear that this Trial setup overcomes many of the
problems of the original Turing Test, even though it still relies on
the subjective judgement of the judge. The problem with the original
Turing Test is that it is just hit-or-miss as to whether the
interrogator asks the right questions that reveal how stupid the
program is. As I said before, people are fooled by ELIZA, but if the
Devil's Advocate asked the questions, he could prove that ELIZA was
stupid in no time---for example, if you just repeat to ELIZA
everything that she says to you, her responses rapidly become
nonsensical. There is no AI program around today that could survive
three rounds of questions from a tough Devil's Advocate.

Daryl McCullough
ORA Corp.
Ithaca, NY
