My research concentrates on demystifying deep learning, and understanding its weaknesses and vulnerabilities.
I work to improve the security, transparency, and generality of deep neural networks, with a focus on applications in data privacy and computer vision.
My work fits primarily under the sub-fields of explainable AI and ML security.
Explainable AI aims to bring interpretability and transparency to otherwise opaque deep learning methods, giving us a richer understanding of their inner workings.
ML security addresses concerns including attacks that compromise data privacy and that fool even state-of-the-art models.
Currently, I am most interested in topics with three major themes; namely,
explaining black-box neural network behavior,
creating a theory of network generalization,
and developing robust and private models.
Explaining Black-box Neural Network Behavior
In the recent years, deep neural networks have become increasingly powerful at tasks previously only humans had mastered.
Deep learning has become widely used, and while it has many practitioners, its inner workings are far from well-understood.
As the application of ML has increased, so has the need for algorithmic transparency, the ability to understand why algorithms deployed in the real world make the decisions they do.
Much of my work has addressed the problem of determining which aspects of a network influence particular decisions, in addition to interpreting the identified influential components.
Influence can be used to increase model trust, to uncover insights discovered by ML models, and as a building block for debugging arbitrary network behavior.
A Theory of Network Generalization
Despite having the capacity to significantly overfit, or moreover, memorize the training data, deep neural networks demonstrate an ability to generalize reasonably well in practice.
Present hypotheses have failed to explain why this is the case.
In fact, it is not well understood how exactly overfitting is manifested in a model.
One aspect of my work tries to understand what phenomena give rise to misclassifications, overfitting, and bias in DNNs.
Understanding the causes for these problems will also shed light on what leads models to generalize; and may suggest ways of improving generalization.
Furthermore, as overfitting presents a threat to the security of a model, understanding overfitting more fundamentally may help protect the privacy of the data involved in training a model, and improve the model's robustness to adversarial manipulation.
I develop explanations for these problems that have direct applications to membership inference, misclassification prediction, and bias amplification.
Robust and Private Models
Deep neural networks have seen great success in many domains, with the ability to master complex tasks such as image recognition, text translation, and medical diagnosis.
Despite their remarkable abilities, neural networks have several peculiar weaknesses.
In particular, there are concerns around the lack of robustness of deep networks to malicious perturbations to their inputs, and around deep networks' tendancy to leak private information about their training data.
My research sheds light on privacy weaknesses in deep networks, paving the way for the development of training routines that ensure privacy without sacrificing the utility of the resuling model.
I am also involved in work towards building models that are not fooled by malicious input perturbations.