Machine Learning Thesis Defense

  • Ph.D. Student
  • Machine Learning Department
  • Carnegie Mellon University
Thesis Orals

Towards Literate Artificial Intelligence

Standardized tests are often used to test students as they progress in the formal education system. These tests are readily available and measurable with clear evaluation procedures and metrics. Hence, it has been proposed that these tests can serve as good benchmarks for AI. In this thesis, we propose approaches for solving some of the common standardized tests taken by students such as reading comprehensions, elementary science exams, geometry questions in the SAT exam and mechanics questions in the AP physics exam. Answering these test problems requires deep linguistic (and sometimes visual) understanding and reasoning capabilities which is challenging for modern AI systems.

In the first half of this thesis, we describe a novel approach to answer natural language comprehension tests such as reading comprehensions and elementary school science tests. These tests evaluate the system's ability to understand text through question-answering tasks on short pieces of text such that the correct answer can be found in the given text. We present new latent structure models for these tasks. We posit that there is a hidden (latent) structure that explains the relation between the question, the correct answer, and the piece of text. Since the structure is latent, it must be inferred. We present a unified max-margin framework that learns to find these hidden structures (given a corpus of question-answer pairs), and uses what it learns to answer machine comprehension questions on novel texts. We also explore a deeper representations of language based on AMR, how can we incorporate external knowledge resources in the latent structure and a simple but effective extension of this framework to incorporate multi-task learning on the different subtasks that are required to perform the overall question answering task.

Reasoning is a key challenge in AI. Thus, in the second half of this thesis, we propose some hard reasoning challenges in the domain of math and science - geometry questions in the SAT exam and mechanics question in the AP physics exam. Solving these tasks requires a deeper ability to incorporate external domain knowledge as well as an ability to perform multi-hop reasoning based on this domain knowledge. We propose a parsing to programs approach for these tasks. P2P can be seen as a natural language interface to expert systems. P2P assumes a formal language for the domain and the domain knowledge written down as rules or programs. This domain knowledge can be manually provided by a domain expert, or, as we show in our work, can be extracted by reading a number of textbooks in an automated way. When presented with a question, P2P learns a representation of the question in the formal language via a multi-modal semantic parser. Then, it uses the formal question interpretation and the relevant domain knowledge to obtain an answer to the question by using probabilistic reasoning.

Thesis Committee:
Eric Xing (Chair)
Jaime Carbonell
Tom Mitchell
Dan Roth (University of Pennsylvania)

Copy of Thesis Document

For More Information, Please Contact: