Deeper QA: CMU, Watson, and the Open Advancement of Question Answering

OAQA provides a foundational architecture and methodology for accelerating collaborative research in automatic question answering. By making long term commitments to an architecture that supports and standardizes simultaneous research experiments, the OAQA is accelerating the advancement of QA technologies. Pioneered by Eric Nyberg and David Ferrucci (IBM), OAQA combines an object-oriented software architecture with comprehensive metrics, measurement and error analysis at the system and module levels. OAQA also defines challenge problems which require different levels of performance along dimensions such as real-time speed, answer accuracy and answer confidence. More details can be found in the OAQA technical report.

CMU has been collaborating with the IBM DeepQA Group since 2007 to develop an open-source framework for OAQA. The initial OAQA architecture and data model were used to build systems for the TREC challenge problem and the Jeopardy! challenge problem. In 2010, Carnegie Mellon students created open-source QA systems for English and Chinese using the OAQA architecture and available open-source components from CMU and elsewhere. Following OAQA principles, teams of current students work to improve these systems as part of their ongoing research and coursework. Carnegie Mellon students Nico Schlaefer and Hideki Shima also contributed algorithms and code to IBM's Watson system, as participants in IBM's summer internship program.

• "What Watson's Win Means"
  Inside Nova, 02/17/11
• "Man versus machine: Chalk one up for the latter in 'Jeopardy!'"
  Pittsburgh Post-Gazette, 02/17/11
• "Computer named Watson drubs humans on 'Jeopardy!'"
  Pittsburgh Tribune-Review, 02/17/11
• "CMU Researcher Outlines How Watson Works"
  International Business Times, 02/16/11
• "IBM's Watson Aims Far Beyond 'Jeopardy' Matchup"
  Investor's Business Daily, 02/12/11
• "CMU and IBM Collaborate on Open Computing System for Advancing Research on Question Answering"
  PR Newswire, 02/11/11
• "IBM Announces Eight Universities Contributing to the Watson Computing System's Development"
  PR Newswire, 02/11/11
• "CMU's Eric Nyberg talks 'Jeopardy' Watson Computer"
  WPXI, 02/11/11
• "Watson Takes On Jeopardy!"
  Carnegie Mellon YouTube, 02/10/11
• "What Happens if Watson Beats Ken Jennings?"
  The Atlantic, 02/07/11
• "Can a Computer Win on 'Jeopardy'?"
  Wall Street Journal, 02/05/11
• "What is Artificial Intelligence?"
  New York Times, 02/05/11
• "Marblehead High grad one of the minds behind 'Jeopardy!' computer"
  Marblehead Reporter, 01/27/11
• "The Jeopardy! Challenge"
  Carnegie Mellon YouTube, 08/31/11
• "Computer Program to Take On 'Jeopardy!'"
  New York Times 04/26/09
• "Towards the Open Advancement of Question Answering"
  IBM Technical Report, 2008
• Eric Nyberg's technical publications on question answering (Google Scholar)
• Eric Nyberg's Google Scholar profile

Roles

• Faculty in the Language Technologies Institute
• Co-director of the Masters of Information Technology:
  Very Large Information Systems (MSIT-VLIS) program
• Co-director of the Masters of Biotechnology Innovation
  and Computing (MSBIC) program
• Consulting Faculty in the MSIT-eBusiness program
  
• Secretary for the UIMA Technical Committee

Courses

• 11-693 Software Methods for Biotechnology
• 11-791 Software Engineering for Information Systems I
• 08-701 Requirements Elicitation
• 08-704 Database and Detailed Design
• 11-792 Software Engineering for Information Systems II
• 08-743 Very Large Information Systems Seminar
• 11-796 Question Answering Lab

Other Projects