Newsgroups: comp.ai
Path: cantaloupe.srv.cs.cmu.edu!rochester!udel!gatech!howland.reston.ans.net!spool.mu.edu!torn!nott!dgbt!rick.dgbt.doc.ca!thom
From: thom@rick.dgbt.doc.ca (Thom Whalen)
Subject: Re: Why I Like the Loebner Competition
Message-ID: <1995Mar15.182755.21375@dgbt.doc.ca>
Sender: news@dgbt.doc.ca (News user)
Organization: Communications Research Centre, Ottawa, Canada
References: <1995Mar8.162130.10752@dgbt.doc.ca> <fano-1103951313070001@fano.ils.nwu.edu> <1995Mar12.224351.7345@dgbt.doc.ca> <korcuska-1403951702420001@korcuska.ils.nwu.edu>
Date: Wed, 15 Mar 95 18:27:55 GMT
Lines: 86

In article <korcuska-1403951702420001@korcuska.ils.nwu.edu> korcuska@ils.nwu.edu (Michael Korcuska) writes:
>
>OK, a session with the winner is below.  I'll be shocked if this is
>anything more than a keyword matching system.  The failure of a keyword
>matching system to engage in intelligent discource will not contribute
>much--the shortcomings of this approach are well known.  Sorry, Thom, as
>long as you're doing information retrieval I'm with you. But as soon as
>you start saying this is making progress towards natural language
>understanding, I can't help but disagree.
>
> [Michael's dialogue fragment deleted]
>

I trust that you will not find the following dialogue fragment too
shocking :-).
------------------------------------------------------------------------------
You say:  How can I make love to another woman?
     Women who are lesbians may kiss each other, masturbate each other, and
engage in oral sex.  Some homosexual women may use vibrators on each other.

You say:  How can I make love to another man?
     Homosexual men may kiss each other, masturbate each other, and engage in
oral sex.  Some homosexual men may engage in anal intercourse, though most do
not do so regularly or frequently.  Many homosexual men have never tried anal
intercourse.

You say:  How can I make love to a woman?
     If you have never tried sexual intercourse before, but only know about it
from books and movies, it can sound rather intimidating.  Fiction pretends that
there are subtle techniques...
	etc.
-------------------------------------------------------------------------------
TIPS/sex produces a different answer for each of the three questions:
	How can I make love to another woman?
	How can I make love to another man?
	How can I make love to a woman?

What would you identify as a keyword in these questions?  There is no single
word that maps to any single answer.

It is interesting to note that these questions also provide examples of some
other things that create problems for linguists.  In the first two sentences,
the program distinguishes between a speaker who is probably a man and a
speaker who is probably a woman because of the implication of the word,
"another".  In the third sentence, the program does not have any clues, and
produces an answer that is as gender-neutral as I could make it.

As another example, you might try giving the following two sentences to
TIPS/sex:
	Can children have sex?
	Can children have sex with each other?

You will find that, in the first case, the program assumes that you mean
children having sex with an adult (presumably you) and warns against
pedophilia.  In the second case, the program realizes that you are asking
about early childhood development and responds appropriately.  But again,
the word "children" is not specific to a single answer (nor are the words,
"with," "each," and "other," as you would see if you use them in other
sentences).

Your comment that TIPS must be a keyword matching system is very interesting
to me, because I get this comment all the time, yet it is not true.  My
standard answer is that language consists of strings of words, so of course
words which are very specific, such as "semen" in your example, are going
to produce an answer that describes semen.  You would find the same thing
if you talk to a human expert.  You would not expect to ask about semen and
get a response about the symptoms of syphilis.  It would be wrong.  People
who assume that TIPS must be a keyword system do not ask what semen
tastes like, if swallowing it could be dangerous, or how much is ejaculated.
If they did, they would see that "semen" does not produce a single answer.

I suspect that the reason that TIPS makes people think of a keyword
matching system is that it contains pre-written answers which are
usually fairly general.  As a result, when people ask related
questions, using the same words, they get the exactly the same answer
back.  This is a dead givaway that they are interacting with a computer
system, and suggests that the answer that they saw is the
only answer which is relevant to the most salient word in their
query.  That this is often the case.  However, it is an error in logic
to generalize to assume that it is always the case.

-- 
   Thomas Whalen, Ph.D.   thom@dgbt.doc.ca   (613) 990-4683       
     Communications Research Centre, Government of Canada
                    http://debra.dgbt.doc.ca/
For a good time, 'telnet debra.dgbt.doc.ca 3000' and ask about sex.
