Machine Learning Seminar

  • Gates Hillman Centers
  • Reddy Conference Room 4405
  • QUANQUAN GU
  • Assistant Professor
  • Computer Science Department
  • University of Virginia
Seminars

Two facets of stochastic optimization: continuous-time dynamics and discrete-time algorithms

Stochastic optimization methods have been widely used in machine learning. However, the convergence analyses of many stochastic optimization methods for both convex and nonconvex optimization remain elusive. In this talk, I will show that the continuous time dynamics can help us better understand stochastic optimization, derive new discrete-time algorithms based on various discretization schemes, and provide a unified and simple proof of convergence rates. More specifically, I will first introduce a new framework based on stochastic differential equations to analyze accelerated stochastic mirror descent algorithms. Under this framework, we providea Lyapunov function based analysis for the continuous-time stochastic dynamics, as well as several new discrete-time algorithms derived from the continuous time dynamics. In the second part of this talk, I will introduce a new analysis of the Langevin dynamics based algorithms for nonconvex optimization, such as gradient Langevin dynamics (GLD) and stochastic gradient Langevin dynamics (SGLD). This analysis is based on analyzing thecontinuous time Langevin dynamics and its discretization error, and leads to faster rates of convergence than existing results.



Quanquan Gu is an Assistant Professor of Computer Science at the University of Virginia. Prior to joining UVa, he was a Postdoctoral Research Associate in the Department of Operations Research and Financial Engineering at Princeton University. He received his Ph.D. degree in Computer Sciencefrom the University of Illinois at Urbana-Champaign. His current research is in the area of statistical machine learning, with a focus on nonconvex statistical optimization and high-dimensional statistical inference. He is a recipient of the NSF CAREER Award (2017) and the Yahoo! Academic Career Enhancement Award (2015).

For More Information, Please Contact: