Abhilasha Ravichander

/ɑ.bʰi.ˈla.ʃə/ (listen)

Abhilasha Ravichander👋

I am a Postdoc/Young Investigator at the Allen Institute for AI (AI2), working on project Mosaic. Prior to joining AI2, I completed my PhD from Carnegie Mellon University. My research focuses on building robust systems to process natural language, by:

(1) formulating techniques to diagnose and validate models and datasets,
(2) developing stronger computational reasoning capabilities, and
(3) explaining the predictive decision-making mechanisms of models.

Ultimately, I care about making trustworthy technology that people can use. To this end, I am also interested in building tools that empower people to take (back) control of their privacy. For more about my work, please see my publications.



What's New

Artifacts or Abduction? was presented at MASC-SLL 2024 and won a best paper award 🎉
I am co-organizing the Workshop on Privacy in Natural Language Processing @ACL 2024
Talk at the National University of Singapore.
Talk at UMass NLP.
I will be at the Rising Stars in EECS workshop at the University of Texas at Austin.
CondaQA won a best paper award at the 2022 SoCal NLP symposium 🎉
I was selected as a Rising Star in Data Science by the University of Chicago.
I was an outstanding reviewer at ACL 2020 and EMNLP 2020.
I started the NLP With Friends online seminar series, with my wonderful co-organizers Zeerak Waseem, Yanai Elazar and Liz Salesky.
I will be speaking in a panel on "The Role of Active Privacy Management in a World Where the Consent Model Breaks Down" at CPDP 2020 in Brussels, Belgium.
I will be helping lead a research team at the OurCS workshop at CMU. If you are an undergraduate woman, please do consider attending! Funds for hotels and meals will be provided.
I will be spending the summer interning at MSR Montreal with Adam Trischler and Kaheer Suleman, working on teaching machines commonsense reasoning.
I will be attending ACL, YRRSDS and SIGDIAL 2018 in Melbourne. Ping me if you'd like to chat!
Our work, "Stress Test Evaluation for Natural Language Inference" was an Area Chair Favorite Paper at COLING 2018!
I will be at the Generalization in Deep Learning workshop at NAACL 2018. Ping me if you'd like to chat!
I will be starting my PhD at the Language Technologies Institute, Carnegie Mellon University in Fall 2018.
I am at the Machine Learning Summer School in Tubingen. Let me know if you'd like to meet up!
Our team was selected to participate in the Alexa Prize with a 100,000$ stipend and additional support from Amazon! Congratulations to all the selected teams.
Our work on "A Persistent Homology Approach to Document Clustering" won the best poster award in 10-701 (Introduction to Machine Learning (PhD)).


Publications

(*) - Equal Contribution

2024


Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question? MASC-SLL 2024 best paper award
Nishant Balepur, Abhilasha Ravichander, Rachel Rudinger
arxiv
PDF Code/Data Long

OLMo: Accelerating the Science of Language Models GeekWire Innovation of the Year award
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hannaneh Hajishirzi
arxiv
PDF Code/Data Long
Press: TechCrunch VentureBeat Forbes GeekWire Axios SD Times Fast Company

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo
arxiv
PDF Code/Data Long
Press: TechCrunch Marktechpost Voicebot

Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, Bill Yuchen Lin
arxiv
PDF Code/Data Long Website
Press: Marktechpost

MacGyver: Are Large Language Models Creative Problem Solvers?
Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ronan Le Bras, Raja Marjieh, Nanyun Peng, Yejin Choi, Thomas L Griffiths, Faeze Brahman
2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2024)
PDF Code/Data Long

What’s In My Big Data?
Yanai Elazar, Akshita Bhagia, Ian Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alane Suhr, Evan Pete Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, Hannaneh Hajishirzi, Noah A. Smith, Jesse Dodge
2024 International Conference on Learning Representations, (ICLR 2024).
PDF Code/Data Long Website
Press: Marktechpost

The Generative AI Paradox: “What It Can Create, It May Not Understand”
Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian Fisher, Abhilasha Ravichander, Khyathi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi
2024 International Conference on Learning Representations, (ICLR 2024).
PDF Long

The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, Yejin Choi
2024 International Conference on Learning Representations, (ICLR 2024).
PDF Code/Data Long Website

Understanding How to Inform Blind and Low-Vision Users about Data Privacy through Privacy Question Answering Assistants
Yuanyuan Feng, Abhilasha Ravichander, Yaxing Yao, Shikun Zhang, Rex Chen, Shomir Wilson, Norman Sadeh
USENIX Security 2024.
PDF Long

2023


Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning
Ximing Lu, Faeze Brahman, Peter West, Jaehun Jang, Khyathi Chandu, Abhilasha Ravichander, Lianhui Qin, Prithviraj Ammanabrolu, Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Yuchen Lin, Skyler Hallinan, Xiang Ren, Sean Welleck, Yejin Choi
2023 Conference on Empirical Methods in Natural Language Processing, (EMNLP 2023).
PDF Code/Data Long

When and Why Does Bias Mitigation Work?
Abhilasha Ravichander*, Joe Stacey*, Marek Rei
Findings of the 2023 Conference on Empirical Methods in Natural Language Processing, (EMNLP Findings 2023).
PDF Code/Data Long


2022


CondaQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation SoCal NLP Symposium best paper award
Abhilasha Ravichander, Matt Gardner, Ana Marasović
2022 Conference on Empirical Methods in Natural Language Processing, (EMNLP 2022).
PDF Code/Data Long

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, Yoav Goldberg
Preprint.
PDF

A Tale of Two Regulatory Regimes: Creation and Analysis of a Bilingual Privacy Policy Corpus
Siddhant Arora, Henry Hosseini, Christine Utz, Vinayshekhar Bannihatti Kumar, Tristan Dhellemmes, Abhilasha Ravichander, Peter Story, Jasmine Mangat, Rex Chen, Martin Degeling, Thomas Norton, Thomas Hupperich, Shomir Wilson, Norman Sadeh
Thirteenth Language Resources and Evaluation Conference, (LREC 2022).
PDF Code/Data Long

2021


Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?
Abhilasha Ravichander, Yonatan Belinkov, Eduard Hovy
16th Conference of the European Chapter of the Association for Computational Linguistics, (EACL 2021).
PDF Code/Data Long

NoiseQA: Challenge Set Evaluation for User-Centric Question Answering
Abhilasha Ravichander, Siddharth Dalmia, Maria Ryskina, Florian Metze, Eduard Hovy, Alan W Black
16th Conference of the European Chapter of the Association for Computational Linguistics, (EACL 2021).
PDF Code/Data Long Website

Measuring and Improving Consistency in Pretrained Language Models
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg
Transactions of the Association of Computational Linguistics, (TACL 2021).
PDF Code/Data Long

Breaking Down Walls of Text: How Can NLP Benefit Consumer Privacy?
Abhilasha Ravichander, Alan W Black, Thomas Norton, Shomir Wilson and Norman Sadeh.
59th Annual Meeting of the Association for Computational Linguistics, (ACL 2021)
PDF Long

2020


On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT
Abhilasha Ravichander, Eduard Hovy, Kaheer Suleman, Adam Trischler, Jackie Chi Kit Cheung.
2020 Joint Conference on Lexical and Computational Semantics, (*SEM 2020).
PDF Code/Data Long

2019


EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference
Abhilasha Ravichander*, Aakanksha Naik*, Carolyn Rose, Eduard Hovy
2019 Conference on Computational Natural Language Learning, (CoNLL 2019).
PDF Code/Data Long

Question Answering for Privacy Policies: Combining Computational and Legal Perspectives
Abhilasha Ravichander, Alan W Black, Shomir Wilson, Thomas Norton and Norman Sadeh.
2019 Conference on Empirical Methods in Natural Language Processing, (EMNLP 2019)
PDF Code/Data Long

Exploring Numeracy in Word Embeddings
Aakanksha Naik*, Abhilasha Ravichander*, Carolyn Rose, Eduard Hovy
57th Meeting of Association for Computational Linguistics, (ACL 2019).                                        
PDF Short

Evaluating How Global Privacy Principles Answer Consumers’ Questions About Mobile App Privacy.
Thomas Norton, Joel Reidenberg, Norman Sadeh and Abhilasha Ravichander
4th European Privacy Law Scholars Conference, (PLSC 2019).

Challenges in Automated Question Answering for Privacy Policies.
Abhilasha Ravichander, Alan Black, Eduard Hovy, Joel Reidenberg, N. Cameron Russell and Norman Sadeh
AAAI Spring Symposium Series, 2019
PDF Long

MAPS: Scaling Privacy Compliance Analysis to a Million Apps
Peter Story, Sebastian Zimmeck, Daniel Smullen, Abhilasha Ravichander, Ziqi Wang, Joel Reidenberg, N. Cameron Russell and Norman Sadeh
PETS 2019
PDF Long

2018


Stress Test Evaluation for Natural Language Inference     Area Chair Favorite Paper Prize
Aakanksha Naik*, Abhilasha Ravichander*, Norman Sadeh, Carolyn Rose, Graham Neubig.
27th International Conference on Computational Linguistics, (COLING 2018).
PDF Code/Data Long Website Slides

An Empirical Study of Self-Disclosure in Spoken Dialogue Systems
Abhilasha Ravichander, Alan Black.
19th Annual SIGdial Meeting on Discourse and Dialogue, (SIGDIAL 2018).                                        
PDFLong


2017


Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology-Based Representations
Paul Michel*, Abhilasha Ravichander*, Shruti Rijhwani*.
Workshop on Representation Learning For NLP, Association for Computational Linguistics, 2017 (ACL 2017).
PDF Short

How Would You Say It? Eliciting Lexically Diverse Data for Supervised Semantic Parsing
Abhilasha Ravichander*, Thomas Manzini*, Matthias Grabmair, Jonathan Francis, Graham Neubig, Eric Nyberg.
18th Annual SIGdial Meeting on Discourse and Dialogue, (SIGDIAL 2017).
PDF Code/Data Long