Newsgroups: comp.ai.doc-analysis.misc,comp.ai.nat-lang
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news4.ner.bbnplanet.net!news.ner.bbnplanet.net!news.mathworks.com!newsfeed.internetmci.com!howland.reston.ans.net!torn!watserv3.uwaterloo.ca!undergrad.math.uwaterloo.ca!news
From: Gord Cormack <gvcormac@plg.uwaterloo.ca>
Subject: Re: Software Knows to Deconstruct in Plain English
Sender: news@undergrad.math.uwaterloo.ca (news spool owner)
Message-ID: <DnBEGw.KJ3@undergrad.math.uwaterloo.ca>
Date: Sun, 25 Feb 1996 04:43:44 GMT
X-Url: http://barley.uwaterloo.ca/gcl/%28%22trec%22%29%3c%28%7b%2d%7d%3c%3e%7b1891389279%7d%29/art/1921124897
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=us-ascii
References: <4ftflg$st9@la1.digilink.net> <4fvchh$bcl@thor.cs.cornell.edu> <4g3656$gpt@la1.digilink.net> <TED.96Feb18222415@hellespont.crl.nmsu.edu> <4gdgnn$d2h@la1.digilink.net>
Nntp-Posting-Host: plg.uwaterloo.ca
Mime-Version: 1.0
X-Mailer: Mozilla 1.1N (X11; I; SunOS 4.1.2 sun4c)
Organization: University of Waterloo
Lines: 21
Xref: glinda.oz.cs.cmu.edu comp.ai.doc-analysis.misc:149 comp.ai.nat-lang:4604

> Subject:    Re: Software Knows to Deconstruct in Plain English
> Author:     reynolds@itpinc.com (Intelligent Text Processing)

As Chris Buckley and others have pointed out, if there were any scientific
basis for the claims of 90+ percent recall and precision, or indeed for the
claims of 42 percent for boolean systems on the same data, ITP should be
prepared to subject its data and methodology to peer review.  TREC is likely
the best forum available.

I am hardly an expert on U.S. trade law, but I thought the FTC had fairly
specific regulations regarding the publication of unsubstantiated quantitative 
claims in advertising.

The demonstration database is hopelessly inadequate.  A worthwhile demonstration
collection would be Internet News, as opposed to the tiny set of AIDS documents now in
place.  Surely ITP has access to a news spool.
 
-- 
Gordon V. Cormack     CS Dept, University of Waterloo, Canada N2L 3G1
gvcormac@uwaterloo.ca           http://cormack.uwaterloo.ca/~gvcormac

