Amanda Bertsch

I am a second-year masters student in the Language Technologies Institute at Carnegie Mellon University, advised by Matt Gormley. I also work closely with members of NeuLab.

I work on summarization and other conditional text generation tasks. My research interests include better ways to reason over large quantities of knowledge, model large-scale structure in text, and effectively integrate external knowledge into models. Currently, my work is in long-doc and multi-doc summarization.

I’m also broadly interested in meta-analysis of the NLP community, including critically examining the benchmarks, datasets, and modeling choices we take as defaults. Right now, some great collaborators and I are interviewing people about paradigm shifts in NLP; if you’ve published 3+ papers in NLP-related venues and you’d have time for a 60 minute interview, please reach out!

Before coming to CMU, I received my bachelors in math and computer science from the University of Arizona, where I was advised by Steven Bethard.

In my spare time, I write and read speculative fiction and play tabletop games.

news

Dec 7, 2022	I’ll be presenting our Findings paper on style transfer for dialogue summarization in the GEM poster session at EMNLP 2022!
Jul 15, 2022	I co-presented work on bias transfer from pretraining datasets at the Gender Bias in NLP workshop at NAACL 2022.
Nov 11, 2021	I presented my undergraduate thesis work on bias detection at the 2021 Workshop on Noisy User-generated Text!

selected publications

Findings

He Said, She Said: Style Transfer for Shifting the Perspective of Dialogues

Bertsch, Amanda, Neubig, Graham, and Gormley, Matthew R.

In Findings of the Association for Computational Linguistics: EMNLP 2022. 2022

Abs PDF

In this work, we define a new style transfer task: perspective shift, which reframes a dialogue from informal first person to a formal third person rephrasing of the text. This task requires challenging coreference resolution, emotion attribution, and interpretation of informal text. We explore several baseline approaches and discuss further directions on this task when applied to short dialogues. As a sample application, we demonstrate that applying perspective shifting to a dialogue summarization dataset (SAMSum) substantially improves the zero-shot performance of extractive news summarization models on this data. Additionally, supervised extractive models perform better when trained on perspective shifted data than on the original dialogues. We release our code publicly.
GeBNLP

Evaluating Gender Bias Transfer from Film Data

Bertsch, Amanda, Oh, Ashley, Natu, Sanika, Gangu, Swetha, Black, Alan W., and Strubell, Emma

In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP) Jul 2022

Abs PDF

Films are a rich source of data for natural language processing. OpenSubtitles (Lison and Tiedemann, 2016) is a popular movie script dataset, used for training models for tasks such as machine translation and dialogue generation. However, movies often contain biases that reflect society at the time, and these biases may be introduced during pre-training and influence downstream models. We perform sentiment analysis on template infilling (Kurita et al., 2019) and the Sentence Embedding Association Test (May et al., 2019) to measure how BERT-based language models change after continued pre-training on OpenSubtitles. We consider gender bias as a primary motivating case for this analysis, while also measuring other social biases such as disability. We show that sentiment analysis on template infilling is not an effective measure of bias due to the rarity of disability and gender identifying tokens in the movie dialogue. We extend our analysis to a longitudinal study of bias in film dialogue over the last 110 years and find that continued pre-training on OpenSubtitles encodes additional bias into BERT. We show that BERT learns associations that reflect the biases and representation of each film era, suggesting that additional care must be taken when using historical data.
W-NUT

Detection of Puffery on the English Wikipedia

Bertsch, Amanda, and Bethard, Steven

In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021) Nov 2021

Abs PDF

On Wikipedia, an online crowdsourced encyclopedia, volunteers enforce the encyclopedia’s editorial policies. Wikipedia’s policy on maintaining a neutral point of view has inspired recent research on bias detection, including “weasel words” and “hedges”. Yet to date, little work has been done on identifying “puffery,” phrases that are overly positive without a verifiable source. We demonstrate that collecting training data for this task requires some care, and construct a dataset by combining Wikipedia editorial annotations and information retrieval techniques. We compare several approaches to predicting puffery, and achieve 0.963 f1 score by incorporating citation features into a RoBERTa model. Finally, we demonstrate how to integrate our model with Wikipedia’s public infrastructure to give back to the Wikipedia editor community.