Machine Learning / Duolingo Seminar

  • Remote Access Enabled - Zoom
  • Virtual Presentation
  • Assistant Professor
  • Robotics Institute, School of Computer Science
  • Carnegie Mellon University

Towards Embodied Intelligence

Learning has led to significant progress in sensorimotor control, however, most successes require training agent policies from scratch for every new robot, task, or environment. Furthermore, trained agents are incredibly specific in that they only perform the tasks they are trained for or the robot hardware they are trained on and are miserable at generalization. This is in contrast to other fields like computer vision or language where learning has been instrumental in pretraining general representations that allow fast adaptation to several downstream tasks. If we are to ever create such general, pre-trainable priors for movement control similar to those for image classification or language modeling, it is imperative for policies to be applicable to a wide variety of robots as well as tasks. But how does one decide what tasks to pretrain for, what robots to pretrain on or how should robot motions be represented so as to generalize well to unseen ones?

In this talk, I will present our initial efforts towards building a framework for learning general-purpose embodied intelligence driven by two key ingredients: curiosity and compositionality. The framework brings together ideas from machine learning, control theory, and developmental psychology to achieve end-to-end sensorimotor learning in embodied agents. I will present results from case studies of robots that achieve strong performance across several simulation benchmarks, tie knots using rope, navigate in office environments, and display drastically diverse locomotion styles across unseen robot shapes.

Deepak Pathak is a faculty in the School of Computer Science at Carnegie Mellon University. He received his Ph.D. in Artificial Intelligence from UC Berkeley and his research spans computer vision, machine learning, and robotics. He is a recipient of the Google Faculty Award, Facebook Graduate Fellowship, the NVIDIA Fellowship, and the Snapchat Fellowship, and his research has been featured in popular press outlets, including The Wall Street Journal, The Economist, Quanta Magazine, Wired, and MIT Technology Review. Deepak received his Bachelor's from IIT Kanpur with a Gold Medal in Computer Science. He founded VisageMap Inc. later acquired by FaceFirst Inc.

Zoom Participation. See announcement.

For More Information, Please Contact: