Newsgroups: comp.ai.nat-lang
Path: cantaloupe.srv.cs.cmu.edu!das-news.harvard.edu!news2.near.net!MathWorks.Com!europa.eng.gtefsd.com!howland.reston.ans.net!cs.utexas.edu!uunet!bcstec!bronte!snake!rwojcik
From: rwojcik@atc.boeing.com (Richard Wojcik)
Subject:  MUC and ARPA (Was: Books on Intro. natural language...)
Message-ID: <1994Oct3.170517.2129@grace.rt.cs.boeing.com>
Sender: usenet@grace.rt.cs.boeing.com (For news)
Reply-To: rwojcik@atc.boeing.com
Organization: Research & Technology
References: <MAGERMAN.94Sep30161015@snoopy.bbn.com>
Date: Mon, 3 Oct 1994 17:05:17 GMT
Lines: 34

I agree with the argument that the MUC competition has tended to favor
statistical methods, but nobody here seems to take into account the type
of message set that ARPA has forced MUCers to get bogged down in.  In my
opinion, ARPA has biased the competition by demanding that people process
extremely unpredictable text--with a lot of metaphorical language to
boot.   NLU methodology will be less likely to succeed under those conditions,
since it places a heavier burden on the developers to anticipate the variation
inherent in the text.  I would think that NLU methodology would do a lot 
better on shorter, more predictable message streams, where the language in
previously unseen text contains less variation.  Unfortunately, ARPA seems to
have decided that "pushing the envelope" on the language would naturally 
lead to greater advances in the ability of such systems to process more mundane
language.  I think that quite the opposite has happened.  It has improved
"dumb" processing methods at the expense of "smart" ones by requiring people
to focus on language that is too rich in complexity.  Nobody is really looking 
at how NLU would fare againsts statistics on MUC2-level language anymore.
(To my knowledge, anyway.)

BTW, I am not complaining about the use of statistical methods.  I think that
they are really great, especially when you need to process highly variable text.
However, it is also worthwhile to process message streams that contain simpler,
more predictable language.  The advantage of the NLU approach is that it
can extract more information out of such streams than the statistical methods
can.  Well, that is my prediction.  I am complaining that it is unfair to criticize
NLU methodology on the basis of a test that is bound to bias the results in
favor of statistical methodology.   The two methodologies are really 
complementary.  They respond to different needs. 

---

Disclaimer:  Opinions expressed above are not those of my employer.

    Rick Wojcik   (rick.wojcik@boeing.com)   Seattle, WA

