Newsgroups: comp.ai
Path: cantaloupe.srv.cs.cmu.edu!rochester!udel!gatech!swrinde!pipex!uunet!psinntp!scylla!daryl
From: daryl@oracorp.com (Daryl McCullough)
Subject: Re: Loebner Prize $2000 and a medal
Message-ID: <1995Apr25.133824.1980@oracorp.com>
Organization: Odyssey Research Associates, Inc.
Date: Tue, 25 Apr 1995 13:38:24 GMT
Lines: 55

marengom@aol.com (MarengoM) wrote:

>We are convinced that no "canned intelligence" can pass the Turing
>test, using either text or A/V, or whatever.  The sphere of human
>knowledge is just too broad to hand-program into a machine.  It's like
>trying to empty the ocean with a pail.  By limiting the I/O
>requirements to text we encourage contesters to continue working on
>useless hacks instead of concentrating on the real important issue
>which is: how to make a machine learn a wide range of knowledge so we
>don't need to spend an eternity hand-feeding it one tiny bit at a
>time.

In comp.ai.philosophy a while ago, I described a version of the Turing
Test which makes a text-based test less susceptible to clever hacks
(although it would be a lot more work for the participants). The idea
is to have four human participants: (1) the judge, who is trying to
decide whether the being in question is intelligent, (2) God's
advocate, who is trying to prove that it is intelligent, (3) the
devil's advocate, who is trying to prove that it is not intelligent,
and (4) a referee, who's job is to make sure that the two advocates
keep within the guidelines. Both sets of advocates and the referee are
given complete access to the computer code, and the
documentation. Based on this documentation, they come up with a line
of questions that they believe will show the program in question
either to be a true AI or a clever hack. The referee's job is to
prevent the advocates from asking leading questions---for example,
questions that could not be truthfully answered by a program without
revealing that it is a program.

"Clever hacks" always work by anticipating the line of questions that
the interviewer will ask, and having canned responses that work in
those cases. Since the devil's advocate is allowed to look at the code
and documentation, he can choose a line of questioning that probes
exactly the weaknesses of the program. If, even knowing how the thing
works, he can't find a question to ask that the program fails to
answer intelligently, then the program is considered intelligent
(or at least not unintelligent).

Daryl McCullough
ORA Corp.
Ithaca, NY














