Sanket Vaibhav Mehta

Sanket Vaibhav Mehta [SVM]

I'm a PhD candidate at the Language Technologies Institute (LTI) at School of Computer Science, Carnegie Mellon University and I'm advised by Emma Strubell.

Previously, I obtained a Master's degree (2019) from the LTI where I was advised by Jaime Carbonell and Barnabás Póczos.

Before joining CMU, I worked as a member of the research staff at Big Data Research Lab, Adobe Research (2015-17) where I worked on designing algorithms for identifying data-driven geo-fences to assist Adobe’s digital marketing offering.

I graduated from Indian Institute of Technology Roorkee with a B.Tech in Computer Science (2011-15) and a President's Gold Medal.

Email / CV / Google Scholar / X / GitHub

Research

I'm interested in machine learning, natural language processing and optimization with a specific focus on learning from limited labeled data, multiple tasks, non-stationary data distributions (Lifelong Learning, Transfer Learning, Meta-Learning, Multi-Task Learning).

Latest News

[Aug. 28, 2023] An Empirical Investigation of the Role of Pre-training in Lifelong Learning will be presented at NeurIPS 2023 Journal-to-Conference Track!

[Jul. 4, 2023] An Empirical Investigation of the Role of Pre-training in Lifelong Learning paper accepted at JMLR 2023!

[Jun. 23, 2023] Conditional Diffusion Replay for Continual Learning in Medical Settings paper accepted at Workshop on Challenges in Deployable Generative AI, ICML 2023!

[Jun. 19, 2023] Adapting to Gradual Distribution Shifts with Continual Weight Averaging paper accepted at Workshop on High-dimensional Learning Dynamics, ICML 2023!

[Jan. 23, 2023] I proposed my doctoral thesis! (Thesis committee: Emma Strubell, William Cohen, Aditi Raghunathan, Dani Yogatama)

[Dec. 19, 2022] DSI++ paper is out! TLDR!

[Nov. 28, 2022] I'm attending NeurIPS'22!

[Oct. 6, 2022] Train Flat, Then Compress paper accepted at Findings of EMNLP 2022!

[July. 10, 2022] An Introduction to Lifelong Supervised Learning primer is out!

[May. 23, 2022] Excited to start summer internship with Yi Tay and Jai Gupta at Google Research!

[Feb. 24, 2022] Compositional Generalization for Data-to-Text Generation paper accepted at ACL 2022!

[Jan. 20, 2022] ExT5: Extreme Multi-Task Scaling paper accepted at ICLR 2022!

Publications

	An Empirical Investigation of the Role of Pre-training in Lifelong Learning Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, Emma Strubell The Journal of Machine Learning Research, 2023 bibtex / code
	DSI++: Updating Transformer Memory with New Documents Sanket Vaibhav Mehta, Jai Gupta, Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Jinfeng Rao, Marc Najork, Emma Strubell, Donald Metzler arXiv, 2022 bibtex / tweet
	Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models Clara Na, Sanket Vaibhav Mehta, Emma Strubell EMNLP Findings, 2022 bibtex / code / tweet
	An Introduction to Lifelong Supervised Learning Shagun Sodhani, Mojtaba Faramarzi, Sanket Vaibhav Mehta, Pranshu Malviya, Mohamed Abdelsalam, Janarthanan Janarthanan, Sarath Chandar arXiv, 2022 bibtex / tweet
	Improving Compositional Generalization with Self-Training for Data-to-Text Generation Sanket Vaibhav Mehta, Jinfeng Rao, Yi Tay, Mihir Kale, Ankur Parikh, Emma Strubell ACL, 2022 bibtex / code / poster
	ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, Donald Metzler ICLR, 2022 bibtex / press / tweet
	Efficient Meta Lifelong-Learning with Limited Memory Sanket Vaibhav Mehta, Zirui Wang, Barnabás Póczos, Jaime Carbonell EMNLP, 2020 bibtex
	Learning Rhyming Constraints using Structured Adversaries Harsh Jhamtani, Sanket Vaibhav Mehta, Jaime Carbonell, Taylor Berg-Kirkpatrick EMNLP, 2019 bibtex / code / poster
	Gradient-Based Inference for Networks with Output Constraints Jay-Yoon Lee, Sanket Vaibhav Mehta, Michael Wick, Jean-Baptiste Tristan, Jaime Carbonell AAAI, 2019 bibtex / code
	Towards Semi-Supervised Learning for Deep Semantic Role Labeling Sanket Vaibhav Mehta, Jay-Yoon Lee, Jaime Carbonell EMNLP, 2018 bibtex / code / poster
	An LSTM Based System for Prediction of Human Activities with Durations Kundan Krishna, Deepali Jain, Sanket Vaibhav Mehta, Sunav Choudhary IMWUT, 2017 bibtex
	Preventing Inadvertent Information Disclosures via Automatic Security Policies Tanya Goyal, Sanket Vaibhav Mehta, Balaji Vasan Srinivasan PAKDD, 2017 bibtex