Large-scale language models (LM) have achieved great results in many NLP applications. However, there is still a non-negligible gap compared with human's capability. One of the key reasons is the lack of an explicit modeling of knowledge. We argue that language models should be equipped with knowledge to better understand entities and relations in the human world. In this talk, I will introduce how to represent and fuse knowledge into language models, which includes three steps: 1) Ground language into related knowledge, 2) Represent knowledge, and 3) Fuse knowledge representation into language model. We demonstrate our work on knowledge-boosted LM in the following work: Dictionary-boosted Language Model, Commonsense Q&A, Open domain QA with Knowledge Graph and Knowledge-text co-pretraining.
Dr. Chenguang Zhu is a Principal Research Manager in Microsoft Cognitive Services Research Group, where he leads the Knowledge & Language Team. His research in NLP covers knowledge graph, text summarization and task-oriented dialogue. Dr. Zhu has led teams to achieve first places in multiple NLP competitions, including CommonsenseQA, CommonGen, FEVER, CoQA, ARC and SQuAD v1.0. He holds a Ph.D. degree in Computer Science from Stanford University.
Faculty Host: Yiming Yang
Zoom Participation. See announcement.