From loebner@ACM.ORG Mon Mar 13 18:32:02 EST 1995 Article: 28091 of comp.ai Path: cantaloupe.srv.cs.cmu.edu!rochester!udel!news.mathworks.com!news.alpha.net!uwm.edu!cs.utexas.edu!natinst.com!hopper.acm.org!ACM.ORG!LOEBNER From: loebner@ACM.ORG (Hugh Loebner) Newsgroups: comp.ai,comp.ai.philosophy Subject: Loebner Prize Rules Date: 13 Mar 1995 17:07:46 GMT Organization: ACM Network Services Lines: 237 Message-ID: <3k1u52$ktd@hopper.acm.org> Reply-To: loebner@ACM.ORG NNTP-Posting-Host: acm.org Xref: glinda.oz.cs.cmu.edu comp.ai:28091 comp.ai.philosophy:25961 LOEBNER PRIZE COMPETITION IN ARTIFICIAL INTELLIGENCE ANNOUNCEMENT OF, AND APPLICATION INFORMATION FOR, THE 1995 PRIZE COMPETITION The Loebner Prize Medal and a cash award is awarded annually to the designer of the computer system that best succeeds in passing a variant of the Turing Test. In 1995, $2,000 and a bronze medal will be awarded to the designer of the Most Human Computer if the median rank of that computer is below the overall median rank of all of the hidden human confederates in the contest. The Loebner Prize Competition in Artificial Intelligence was established in 1990 by Dr. Hugh Loebner and the Cambridge (Massachusetts) Center for Behavioral Studies. In accordance with the requirements of Dr. Loebner as published in the June 1994 Communications of the ACM, the winner of the Grand Prize must be prepared to deal with audio visual input (pattern recognition). However, to celebrate the first unrestricted Turing Test, this rule will be waived for the 1995 contest only. Should the median rank of a computer equal or exceed the overall median rank of the confederates -- in other words, should that computer prove to be indistinguishable from the typical human in the event -- the Grand Prize of $100,000 and a gold medal will be awarded, and the contest will be discontinued. The highest-ranking human confederate will receive an award for being the Most Human Human. Applications for the 1995 competition must be postmarked by November 1, 1995. Submissions will be evaluated and no more than eight finalists selected by November 21, 1995. The 1995 finalists will compete in a simultaneous and real-time competition on Friday, December 15, 1995, in New York NY (date and location subject to change). There is no entry fee. Applications must be accompanied by printed or e-mbail protocols recording interactions between the computer system to be entered and one or more human subjects. Protocols must not exceed ten typewritten double-spaced pages or 2,500 words. Applications may be submitted by e-mail. The Cambridge Center or the Prize Committee may request an opportunity to interact with candidate programs. Transcripts and scores from the 1991, 1992, 1993, and 1994 competitions may be obtained by contacting the Cambridge Center for Behavioral Studies at 617-491-9020 (e-mail 76557.1175@compuserve.com). Please review the accompanying rules carefully and provide the following information: Name of Entrant: Professional Title of Entrant (if applicable): Affiliation (if applicable): Contact Person (if applicable): U S Postal Address: Telephone Number(s): Fax Number(s): E-Mail Address(es): Computer hardware to be employed: Computer software to be employed (optional): Enclosed is a protocol recording interactions between the computer system to be entered and one or more human subjects (maximum 10 typewritten double-spaced pages or 2,500 words). Mail completed applications to: Hugh Loebner, Project Director Loebner Prize Competition c/o Crown Industries, Inc. 155 North Park St. East Orange, NJ 07017 e-mail address: loebner@acm.org ============================================ OFFICIAL RULES 1995 LOEBNER PRIZE COMPETITION IN ARTIFICIAL INTELLIGENCE ============================================ 1. The objective of the 1995 Loebner Prize Competition in Artificial Intelligence is to identify the computer system that can best succeed in passing a modern variant of the Turing Test. Judges will attempt to distinguish computer systems (referred to henceforward as "contestants") from human beings (referred to henceforward as "confederates" or "human confederates") based on interactions with these computer entries. 2. Applicants may be individuals, organizations, businesses, schools, corporations, institutions, or other entities. Individuals need not have institutional affiliations. Applicants may be of any nationality or age. Applicants may submit more than one entry during the same year, but a separate application must be completed for each entry. 3. Employees, affiliates, and agents of the Cambridge Center for Behavioral Studies, Crown Industries, Inc., or of other official contest sponsors, as well as members of the families of such individuals, are not eligible to apply. 4. Applications for the 1995 Prize competition may be made on an official application form or facsimile or submitted by electronic mail. Applications must be postmarked by November 1, 1995. Applications should be submitted to: Dr. Hugh Loebner at the above address 5. If there are two or more entries there will be a Loebner Prize contest. The medal and cash award will go to the designer of the computer system with the highest median score, based on the ratings of the contest judges. Mean scores will be used only to break a tie. The Grand Prize of $100,000 and a gold medal will be awarded to the designer of the highest-ranking computer if the median rank of that computer equals or exceeds the median rank of the hidden human confederates. If there is only one entry, Loebner Prize Medal and the $2000.00 cash award will be awarded to that entry. If there are no entries in the 1995 Loebner Prize Competition, the $2000.00 cash prize will be added to the cash award for 1996, making the award for 1996 $4000.00. 6. It is the task of the computer entries to respond to the communications of the judges in such a manner as to imitate the responses of a human. No constraints will be placed on the judges' communications. Entries must be prepared to communicate for an indefinite period of time. 7. Computer entries may contain standard or customized software and hardware. The hardware may be of any type as long as it is inorganic and as long as its replies are in no manner controlled by human beings or other organic systems in real time. 8. Entries will be required to run on hardware located at the competition site. No telecommunications will be permitted. Finalists chosen to participate may submit their entries as programs recorded on standard machine readable media (magnetic or optical storage) together with operating documentation. Macintosh and IBM compatible computers with standard mass storage peripherals will be available on-site for the contest. Efforts will be made to secure the use of other standard computers such as workstations, but this can not be guaranteed. Entrants requiring non-standard hardware will be required to provide that hardware for operation at the contest site. Personnel will be prepared to operate the computer entries according to documentation providedby the contestant. 9. Appropriate steps will be taken to prevent the unauthorized duplication or publication of contestants' entries, however neither the Cambridge Center, the Prize Committee, nor any of their agents can guarantee absolute security. Contestants, by entering this contest, imply their understanding of this and agree to hold the Cambridge Center, the Prize Committee and their agents harmless should there be any unauthorized duplication or publication of programs. Entrants who wish absolute security will be allowed to to operate their entries and/or provide their own hardware on site. 10. Computer entries will be required to record the conversations as text files on magnetic media. Information regarding the file format will be available after April 1, 1995 from the Contest Director. The recordings will remain the property of the Cambridge Center, which will also retain the copyright on transcripts or other representations, magnetic or otherwise, of the recordings. 11. Applications must be accompanied by printed protocols or e-mail transcripts recording actual interactions between the system to be entered and one or more human beings. The protocols may not exceed ten double-spaced typewritten pages or 2,500 words. 12. The selection process may entail interaction between selection personnel and computer entries. Applicants will be notified of the selection decision by November 21, 1995. No more than eight entries will be selected as finalists to compete in a real- time and simultaneous contest to be held in New York, NY, on Friday, December 15, 1995 (date and location subject to change). 13. Judges and human confederates for the 1995 competition will be selected who have no prior knowledge of any of the contestants or human confederates. 14. Judges will have one or more opportunities to interact with each of the computer terminals available concurrently during the contest. Judges will be allowed unrestricted communications. They will be informed that at least two of the terminals are controlled by human confederates and that at least two of the terminals are controlled by computers. 15. One or more referees may be present to limit communications of the confederates. 16. Judges will not be allowed to interact with each other and will be instructed to provide individual ratings of each of the appropriateness and responsiveness of the replies. 17. The names "Loebner Prize" and "Loebner Prize Competition" may be used by contestants in advertising only by advance written permission of the Cambridge Center, and their use may be subject to applicable licensing fees. Advertising is subject to approval by representatives of the Loebner Prize Competition. Improper or misleading advertising may result in revocation of the prize and/or other actions.