CMU CMU Artificial Intelligence Seminar Series sponsored by Fortive Fortive


Back to Seminar Schedule

Wednesday, Mar 24, 2021

Time: 01:00 - 02:00 PM ET
Recording of this Online Seminar on Youtube

Le Song -- Understanding Deep Architectures with Reasoning Layer

Relevant Paper(s):

Abstract: Recently, there has been a surge of interest in combining deep learning models with reasoning in order to handle more sophisticated learning tasks. In many cases, a reasoning task can be solved by an iterative algorithm. This algorithm is often unrolled, and used as a specialized layer in the deep architecture, which can be trained end-to-end with other neural components. Although such hybrid deep architectures have led to many empirical successes, the theoretical foundation of such architectures, especially the interplay between algorithm layers and other neural layers, remains largely unexplored. In this paper, we take an initial step towards an understanding of such hybrid deep architectures by showing that properties of the algorithm layers, such as convergence, stability and sensitivity, are intimately related to the approximation and generalization abilities of the end-to-end model. Furthermore, our analysis matches closely our experimental observations under various conditions, suggesting that our theory can provide useful guidelines for designing deep architectures with reasoning layers.

Bio: Le Song is a Professor and the Deputy Chair of the Machine Learning Department, Mohamed Bin Zayed University of AI, UAE. He was an Associate Professor in the Department of Computational Science and Engineering, College of Computing, Georgia Institute of Technology, worked as a research scientist at Google, and did his post-doc in Carnegie Mellon University. His principal research area is machine learning, especially kernel methods, deep learning, and probabilistic graphical models. He is the recipient of many best paper awards at major machine learning conferences, such as NeurIPS, ICML and AISTATS, and the NSF CAREER Award. He has also served as the area chair or senior program committee for many leading machine learning and AI conferences such as NeurIPS, ICML, ICLR, AAAI and IJCAI, and the action editor for JMLR and IEEE TPAMI. He is also a board member of the International Conference for Machine Learning.