Newsgroups: comp.ai
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!spool.mu.edu!torn!nott!dgbt!rick.dgbt.doc.ca!thom
From: thom@rick.dgbt.doc.ca (Thom Whalen)
Subject: Re: Why I Like the Loebner Competition
Message-ID: <1995Mar12.224351.7345@dgbt.doc.ca>
Sender: news@dgbt.doc.ca (News user)
Organization: Communications Research Centre, Ottawa, Canada
References: <1995Mar8.162130.10752@dgbt.doc.ca> <fano-1103951313070001@fano.ils.nwu.edu>
Date: Sun, 12 Mar 95 22:43:51 GMT
Lines: 114

> I have included my dialog with your system.
> Could you explain which of Eliza's tricks you eschewed and what
> significance you think your approach has?
> The stated purpose of the competition is to "to further the scientific
> understanding of complex human behavior."
> As the winner, do you feel your work in connection with this competition
> has furthered the scientific understanding of complex human behavior?

Sure.  First, though, let me point out that the dialog that you
included is with the much older AIDS Information System.  The Sex
Information system won the Loebner contest.  It is distinguished from
the AIDS system by a newer and more complex technology, as well as
being an attempt to attack a broader and more colloquial subject
matter.

The main distinction in conversations with TIPS (or my older CHAT
system) is that the computer supplies large amounts of information in
its side of the conversation.  Eliza, and other similar programs, do
not.  In their conversations, they simply reflect the human side of the
conversation back to the person.  To quote Weizenbaum, "...ELIZA was an
actress who commanded a set of techniques but who had nothing of her
own to say." [Computer Power and Human Reason, W. H. Freeman and Co.,
Page 188].  As you can see in the dialogue that you quoted, CHAT had a
lot to say.

This was necessary for the objective of my research, which is not to
simulate intelligence, but merely to make it easy for people to get
information from a computer.  As it happens, the most natural way for
people to get information is through a natural language dialogue and
that tends to produce computer-based information systems that behave a
little bit like people; more like people than some other AI programs,
according to the Loebner judges.

In order to achieve this objective, which the opposite of Eliza's, I
had to use an approach which was the opposite of Eliza's.  Eliza is
basically a collection of syntactic rules.  TIPS contains no syntactic
rules at all.  Rather it contains a collection of templates which are
used in a fairly complex search algorithm.  One subtle benefit of this
grammer-free approach is that it is language-free.  We have translated
the AIDS information system into French and it performed well.  We did
not have to develop a French grammer.  We were also able to license the
technology to a Korean company to use in Korea for exactly the same
reason.

Clearly, I am not attempting to achieve natural language
understanding.  We refer to my approach as "natural language control"
because it controls the delivery of information without understanding
it.  This means that we will never solve any problems, with this
technology, which require true understanding (such as machine
translation).

Another interesting difference from Eliza which results from this
approach is that the templates of sentences and the pre-written
responses are specific for a conversation.  Unlike Eliza in which all
of the effort was dedicated to writing the software, writing a natural
language conversation with TIPS requires a lot of writing in English.
In many ways, I view writing a natural language conversation as more
akin to writing a novel or textbook than writing a computer program.

If you compare their conversations, you can easily see the differences
between TIPS and Eliza.  TIPS often provides an inappropriate answer,
but never provides a meaningless one (unless I have written
a meaningless paragraph).  Eliza often generates sentences which are
both meaningless and do not conform to English grammer.

TIPS never says, "Tell me more about X," or "What else comes to mind
when you think of X?" or "What makes you think X?" where X is some
fragment of the human's last utterance.  In many ways, Elisa works
harder to keep the conversation alive than TIPS.  Because TIPS is a
very general system to support a human-authored conversation, it would
be fairly easy to write answers in a TIPS conversation which are
similar to the answers that Eliza provides but I deliberately avoid
writing such answers into my conversations because I do not like the
trickery that it implies.  Elisa never says, "I do not understand.",
but TIPS says that frequently.  It is consistant with my overall
objective to provide information easily and naturally to have my
conversation admit that it does not understand than to pretend that it
does.

As far as your question about whether this "furthers the scientific
understanding of complex human behaviour," I would say it furthers my
understanding.  It remains to be seen whether anyone else can learn
its lessons as well.  The primary lesson that I am learning is that,
by avoiding linguistic competence and addressing actual performance
(i.e., avoiding linguistics and concentrating on real behaviour, I
am, after all, a pschologist and not a linguist or computer scientist),
natural language conversations are not always as regular as they
appear.  I can speculate that a natural language, like English, is
not determined by a fixed, formal grammer, but may be constructed from
a very large set of heuristic rules.  These rules may be similar to one
another, but need not be logically consistant.  They may differ from
one conversational topic to another, from one speaker to another, and
from one context to another.

In the long run, I expect to find the descendants of TIPS converging
with the descendants of Brook's subsumption architectures rather than
with the descendants Chompsky's mathematical grammers.

In 1972, in one of Don Norman's seminars at UCSD, he said that if
anyone could show him a computer program which conversed in natural
language, he could learn something from it, no matter what technique it
used.  I would like, some day, to show him such a program.

-----------------------------------------------------------------------

To see the 1994 Loebner winner:
        telnet debra.dgbt.doc.ca 3000

and ask about sex.

-- 
   Thomas Whalen, Ph.D.   thom@dgbt.doc.ca   (613) 990-4683       
     Communications Research Centre, Government of Canada
         3710 Carling Avenue, Ottawa, Canada  K2H 8S2
