Today, machine learning is transitioning from research to widespread deployment. This transition requires algorithms that can learn from heterogeneous datasets and models that can operate in complex, often multitask settings. So, is there a set of principles we could follow when designing models and algorithms for such settings?
The ideas presented in this thesis are organized as follows. First, we introduce our core probabilistic multitask modeling framework. Starting with a general definition of a learning task, we show how multiple related tasks can be assembled into and represented by a joint probabilistic model. We then define different notions of generalization in multitask settings and demonstrate how to derive practical learning algorithms and consistent objective functions that enable certain types of generalization using techniques from probabilistic learning and inference. Next, we illustrate our proposed framework through multiple concrete case studies in federated learning, reinforcement learning in nonstationary environments, multilingual machine translation, and interpretability of predictive models. Each of our case studies is an independent vignette that focuses on a particular domain and showcases the versatility of our framework. Not only we reinterpret different problems from a probabilistic standpoint, but we also develop new learning algorithms and inference techniques that improve upon the current state-of-the-art in the considered domains.
Eric Xing (Chair)
Pieter Abbeel (University of California, Berkeley)
Rich Caruana (Microsoft Research)
Zoom Participation. See announcement.