Newsgroups: comp.ai.philosophy
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!news.mathworks.com!zombie.ncsc.mil!news.duke.edu!godot.cc.duq.edu!newsfeed.pitt.edu!gatech!swrinde!sgiblab!wetware!spunky.RedBrick.COM!psinntp!psinntp!psinntp!scylla!daryl
From: daryl@oracorp.com (Daryl McCullough)
Subject: Re: Bag the Turing Test
Message-ID: <1994Dec7.153905.11481@oracorp.com>
Organization: Odyssey Research Associates, Inc.
Date: Wed, 7 Dec 1994 15:39:05 GMT
Lines: 35

Several people have complained in this group that a purely behavioral
notion of intelligence doesn't distinguish between real thinking and
memorization. It is certainly the case in schools that we want
children to learn how to solve problems, and not simply to memorize
the answers.

However, we need to ask *why* we care about the difference. I think
that the only reason that people are suspicious of memorization in
nonphilosophical settings is the belief that *eventually* the person
who memorizes will be tripped up---there will be some situation that
he won't have a ready answer for. Therefore, knowing something about
*how* a person gets his answers gives us a clue as to the quality of
*future* answers. This was a point that I made to Jeff Dalton a couple
of years ago. Even if you are only interested in external behavior,
knowing *how* that behavior is produced is useful, since that gives
you clues as to when the behavior will break down. For example, if you
know that ELIZA produces its output by paraphrasing its last input,
then you can figure out how to make ELIZA spew out clearly
nonintelligent garbage.  (For example, try simply repeating to ELIZA
exactly what she says to you.)

So, I think that a Turing Test can actually be quite robust if you
interpret "passing the test" as "giving sensible answers to all
possible inputs". I think that if there is some cheap method to
achieve convincing outpus, then there will exist a line of questioning
that will reveal the limitations of that method. If we ignore the
controversial case of a Humongous Lookup Table (which is not
realizable in practice), then there is no known "cheap" way of passing
the Turing Test (in my strong sense).

Daryl McCullough
ORA Corp.
Ithaca, NY


