CMU Artificial Intelligence Seminar Series ^{sponsored by}

Tuesday, March 28, 2023

Time: 12:00 - 01:00 PM ET

Lucio Dery -- An automated transfer learning approach to tackling learning under limited data

Relevant Paper(s):

• N/A

Abstract: Transfer learning is arguably the engine of the current deep learning revolution in machine learning. A common branch of transfer learning is learning with auxiliary objectives — supplementary learning signals that are introduced to help aid learning on data-starved or highly complex end-tasks. Whilst much work has been done to formulate useful auxiliary objectives, their construction is still an art which proceeds by slow and tedious hand-design. Intuition for how and when these objectives improve end-task performance has also had limited theoretical backing.

In this talk, I will present a task agnostic approach for automatically generating a suite of auxiliary objectives and maximally utilizing them to benefit a specified end-task. We achieve this by deconstructing existing objectives within a novel unified taxonomy, identifying connections between them, and generating new ones based on the uncovered structure. We theoretically formalize widely-held intuitions about how auxiliary learning improves generalization on the end-task which leads us to a principled and efficient algorithm for searching the space of generated objectives to find those most useful to a specified end-task.