TOP-DOWN NEARLY-CONTEXT-SENSITIVE PARSING

EUGENE CHARNIAK

Computer Science Department, Brown University

We present a new syntactic parser that works left-to-right and top-down,
thus maintaining a fully connected parse tree for a few alternative parse
hypotheses. All of the commonly used statistical parsers use context-free
dynamic programming algorithms and as such work bottom up on the entire
sentence. Thus they only find a complete fully connected parse at the very
end. In contrast, both subjective and experimental evidence shows that
people understand a sentence word-to-word as they go along, or close to
it. The constraint that the parser keeps one or more fully connected
syntactic trees is intended to operationalize this cognitive fact. Our
parser achieves a new best result for top-down generative parsers of
89.4%, a 20% error reduction over the previous result for parsers of this
type of 86.8% (Roark, 2001). The improved performance is due to embracing
the very large feature set available in exchange for giving up dynamic
programming.

BIO

Eugene Charniak is University Professor of Computer Science and Cognitive
Science at Brown University and past chair of the Department of Computer
Science. He received his A.B. degree in Physics from University of
Chicago, and a Ph.D. from M.I.T. in Computer Science.  He has published
four books, the most recent being Statistical Language Learning. He is a
Fellow of the American Association of Artificial Intelligence and was
previously a Councilor of the organization. His research has always been
in the area of language understanding or technologies which relate to it.
Over the last 20 years he has been interested in statistical techniques
for many areas of language processing, including parsing and discourse.