In "The Hobbit", J.R.R. Tolkien tells the tale of Bilbo Baggins being whisked away from the comfort of the Shire on an adventure involving dragons and treasure.  This life-changing experience enriched his character and irrevocably altered his world view.

In 2010, I started on my own adventure: leaving the comforts of the Ivory Tower, I joined Twitter, which at the time was a frantic, fast-growing company stumbling and bumbling around in the new topsy-turvy world of social media and big data.  I worked on analytics infrastructure to support data science and services designed to connect users to relevant content.  In the following two years, Twitter's Hadoop data warehouse grew from dozens of nodes to tens of thousands of nodes across multiple datacenters, ingesting tens of terabytes daily from dozens of heterogeneous sources.  Beyond analytics infrastructure, I had the opportunity to contribute to a variety of data products, including real-time search and graph-based recommendations.

Eventually, I made it back to the comforts of the academic Ivory Tower, and like Bilbo Baggins, the experience has profoundly shaped my core identity as a hobbit (errrr, I mean researcher).  In this talk, I will attempt to distill my experiences working on "big data" systems into a series of high-level "lessons learned", using graph-based recommendations as a backdrop.  I'll attempt to connect my experiences in industry back to academic research and discuss my views about the evolving roles of academia and industry.

Professor Jimmy Lin holds the David R. Cheriton Chair in the David R. Cheriton School of Computer Science at the University of Waterloo.  He graduated with a Ph.D. in Electrical Engineering and Computer Science from MIT in 2004.  Lin's research aims to build tools that help users make sense of large amounts of data.  His work lies at the intersection of information retrieval and natural language processing, with a focus on large-scale distributed algorithms and infrastructure for data analytics.  From 2010-2012, Lin spent an extended sabbatical at Twitter, where he worked on services designed to connect users to relevant content and analytics infrastructure to support data science.  Experience in academia and industry guides him in building useful applications that solve real-world user problems while addressing fundamental challenges in computer and information science.

Faculty Host: Jamie Callan


I will talk about two problems in semantic inference.  First, I will describe a method for parsing natural language questions into logical forms, that can be mapped to information stored in structured knowledge bases.  The method relies on deterministic mapping of syntactic dependency trees to logical forms, that could be in turn used for knowledge base inference.  Our approach is inherently multilingual, only relying on automatic dependency parses.  The second problem I will focus on is natural language inference (NLI).  Here, the goal is to determine whether two sentences entail or contradict each other, or has no relationship.  I will present a new "decomposable neural attention model", that is easily parallelizable on new computer architectures such as GPUs, and reaches state-of-the-art results on a recent NLI dataset, but using almost an order of magnitude less model parameters than previous work.

Dipanjan Das is a Senior Research Scientist at Google focusing on learning semantic representations of language.  He received a Ph.D. in 2012 from the Language Technologies Institute, School of Computer Science at Carnegie Mellon University.  Before that, he completed an undergraduate degree in Computer Science and Engineering in 2005 from the Indian Institute of Technology, Kharagpur.  His work on multilingual learning of sequence models received the best paper award at ACL 2011 and a best paper award honorable mention at EMNLP 2013.

Adobe Research Labs would like to invite you to an interactive session and overview of various research opportunities at the Adobe Research Big Data Experience Lab (Bangalore and San Jose) around AI, Machine Learning, Natural Language Processing, Distributed Systems and more.

Advances in Big Data technologies has revolutionized every aspect of modern businesses. As this digital revolution in enterprises gains traction across the world, the confluence of big data with mobile and IoT (Internet of Things) is well positioned to revolutionize how we live and how modern societies operate. Algorithms and software we develop for mobile and IoT are increasingly becoming device, user, social-network, community and context centric. In this talk I would like to introduce a family of algorithms problem patterns that enable new generation of intelligent services. These services help in creating next generation mobile, content and IoT applications that are powered off of big data driven cyber-intelligence in the cloud. In particular we would like to highlight on location, user and content intelligence algorithms through examples, and if time permits do live demonstration of some initial applications of the technology.

Early in the talk we will also provide a brief overview of the research work being done at the Adobe’s Big Data Experience Labs.

Dr. Shriram Revankar is a Fellow and Vice President at Adobe Systems and he heads Adobe’s Big Data Experience Labs. Previously he headed the Adobe’s Research labs in India. Before joining Adobe, Dr. Revankar was with Xerox Research, where he was a Xerox Fellow and the head of Smart & Adaptive Systems Lab. In his current capacity Shriram is focused on developing world class competency in big data, social network analysis, data mining and big-data analytics, natural language processing, knowledge management, machine learning and related areas. Shriram has a M.S. and Ph.D. in Computer Science from SUNY Buffalo, and his research interests span smart & adaptive systems, computer vision & image processing, and analysis of social networks & big data.

Dr. Balaji Vasan Srinivasan is a Computer Scientist at the Adobe Research Big data Experience Labs, Bangalore, India. His research interests span the areas of data mining, natural language processing, machine learning, social data analytics and high performance computing. He finished his Ph.D. in Computer Science at the University of Maryland in September 2011, his thesis was on Scalable Learning Methods for Speaker Recognition and Geostatistics. His research experience also includes stints at National Institutes of Health, Bethesda, MD (May – Aug 2007) and Xerox Research Center, Webster, NY (May – Aug 2011).

Advances in neural network architectures and training algorithms have demonstrated the effectiveness of representation learning in natural language processing. This thesis stresses the importance of computationally modeling the structure in language, even when learning representations.

We propose that explicit structure representations and learned distributed representations can be efficiently combined for improved performance over (i) traditional approaches to structure and (ii) uninformed neural networks that ignore all but surface sequential structure. We demonstrate on three distinct problems how assumptions about structure can be integrated naturally into neural representation learners for NLP problems, without sacrificing computational efficiency.

First, we introduce an efficient model for inferring the phrase-structure trees using given dependency syntax trees as constraints and propose to extend the model, making it more expressive through non-linear (neural) representation learning.

Second, we propose segmental recurrent neural networks (SRNNs) which define, given an input sequence, a joint probability distribution over segmentations of the input and labelings of the segments and show that comparing to models that do not explicitly represent segments such as BIO tagging schemes and connectionist temporal classification (CTC), SRNNs obtain substantially higher accuracies.

Third, we consider the problem of Combinatory Categorial Grammar (CCG) supertagging. We propose to model the compositionality both inside these tags and between these tags. This enables the model to handle an unbounded number of supertags where structurally naive models simply fail.

The techniques proposed in this thesis automatically learn structurally informed representations of the inputs. These representations and components in the models can be better integrated with other end-to-end deep learning systems within and beyond NLP.

Thesis Proposal Committee:
Noah Smith (Co-Chair, CMU/University of Washington)
Chris Dyer (Co-Chair)
Alan Black
Michael Collins (Columbia University)

Copy of Proposal Document

More and more of life is now manifested online, and many of the digital traces that are left by human activity are in natural-language format.  In this talk I will show how exploiting these resources under a computational framework can bring a new understanding of online social dynamics; I will be discussing three of my efforts in this direction.

The first project explores the relation between users and their community, as revealed by patterns of linguistic change.  I will show that users follow a determined life-cycle with respect to their susceptibility to adopt new community norms, and how this insight can be harnessed to predict how long a user will stay active in the community.

The second project proposes a computational framework for identifying and characterizing politeness, a central force shaping our communication behavior.  I will show how this framework can be used to study the social aspects of politeness, revealing new interactions with social status and community membership.

I will conclude by showing that conversational patterns can be predictive of the future evolution of a dyadic relationship.  In particular, I will characterize friendships that are unlikely to last and examine temporal patterns that foretell betrayal in the context of the Diplomacy strategy game.

This talk includes joint work with Jordan Boyd-Graber, Dan Jurafsky, Srijan Kumar, Jure Leskovec, Vlad Niculae, Christopher Potts, Moritz Sudhof and Robert West.

Cristian Danescu-Niculescu-Mizil's research aims at developing computational frameworks that can lead to a better understanding of human social behavior, by unlocking the unprecedented potential of the large amounts of natural language data generated online.   He is the recipient of several awards, including the WWW 2013 Best Paper Award and a Yahoo! Key Scientific Challenges award, and his work has been featured in popular-media outlets such as the New Scientist, NBC's The Today Show, NPR and the New York Times.

Faculty Host: Carolyn Rose
Instructor: Alexander Hauptmann

Interest in creating KBs has often been motivated by the desire to support reasoning on information that would otherwise be expressed in noisy free text and spread across multiple documents.  However, distilling knowledge into a restricted KB can lose important semantic diversity and context.  Traditionally a KB has a single hand-designed schema of entity- and relation-types.  In contrast, universal schema operates on the union of many input schemas, including a great diversity of free textual expressions.  We learn to generalize across these many inputs using deep learning---embedding relation expressions in a common semantic vector space.  In this talk I will introduce universal schema, then describe recent work (a) having the textual entity- and relation-mentions themselves represent the KB, (b) using universal schema, RNNs and neural attentio McCallum's web page is n models to provide generalization, (c) performing logical reasoning on top of this text-embedding-KB, and (d) future work on reinforcement learning to guide the search for proofs of the answers to queries.

Andrew McCallum is a Professor and Director of the Information Extraction and Synthesis Laboratory, as well as Director of Center for Data Science in the College of Information and Computer Science at University of Massachusetts Amherst. He has published over 250 papers in many areas of AI, including natural language processing, machine learning and reinforcement learning; his work has received over 45,000 citations. He obtained his PhD from University of Rochester in 1995 with Dana Ballard and a postdoctoral fellowship from CMU with Tom Mitchell and Sebastian Thrun.  In the early 2000's he was Vice President of Research and Development at at WhizBang Labs, a 170-person start-up company that used machine learning for information extraction from the Web.

He is a AAAI Fellow, the recipient of the UMass Chancellor's Award for Research and Creative Activity, the UMass NSM Distinguished Research Award, the UMass Lilly Teaching Fellowship, and research awards from Google, IBM, Microsoft, and Yahoo. He was the General Chair for the International Conference on Machine Learning (ICML) 2012, and is the current President of the International Machine Learning Society, as well as member of the editorial board of the Journal of Machine Learning Research.

For the past ten years, McCallum has been active in research on statistical machine learning applied to text, especially information extraction, entity resolution, social network analysis, structured prediction, semi-supervised learning, and deep neural networks for knowledge representation. His work on open peer review can be found at

Smart objects are ordinary objects that have been augmented with computation and communication as well as abilities for perception, action and/or interaction.  To be accepted as intelligent, a smart object should behave in a manner that is appropriate for its role and its environment. Appropriate behavior is commonly referred to as situated. In this talk we will discuss the use of Situation Models as theory for constructing smart objects capable of situated interaction.

We describe a layered architecture, in which the situation model is used to coordinate perception, action and interaction. We describe the components of a situation model, and discuss construction of situation models using both GUI based tools and machine learning.  We present a cyclic process inspired by models from ergonomics for maintaining a situation model, and discuss the use of probabilistic predicates for reasoning with uncertain information.  We present examples of applications from the EU projects FAME and CHIL as well as situated interaction with robots, and recent commercial applications proposed by INRIA startups Rocamroll and Situ8ed.

James Crowley holds the post of Professor, Classe Exceptionelle 2, at the Institut National Polytechnique de Grenoble (INPG), where he teaches courses in Computer Vision, Signal Processing, Pattern Recognition, Machine Learning and Artificial Intelligence at ENSIMAG.  He performs his research at INRIA Grenoble Rhône- Alpes Research Center in Montbonnot, where he directs the INRIA  Pervasive interaction Project-Group.

Over the last 30 years, professor Crowley has made a number of fundamental contributions to computer vision and mobile robotics.  In September 2011, James Crowley was appointed as Senior Member of the l'Institut Universitaire de France (IUF).  In Mar. 2014, James Crowley was named Chevalier de l'Ordre National du Mérite.  During his career, Professor Crowley has edited two books, five special issues of journals, and authored over 220 peer-reviewed scientific articles on computer vision, mobile robotics, human-computer interaction and ambient intelligence.  His publications have received over 11,000 Citations and an h-index of 51.

Ph.D. CMU, 1981; MSc CMU, 1977; BSc, SMU, 1975

Faculty Host: Florian Metze
Instructor: Alex Hauptmann


One of the characteristics of spontaneous speech that distinguishes it from written text is the presence of disfluencies, including filled pauses (um, uh), repetitions, and self corrections. In spoken language processing applications, disfluencies are typically thought of as "noise" in the speech signal.  However, there are several systematic patterns associated with where disfluencies occur that can be leveraged to automatically detect them and to improve natural language processing.   Further, rates of different types of disfluencies appear to depend on multiple levels of speech production planning and to vary depending on the individual speaker and the social context.   Thus, detecting different disfluency types provides additional information about spoken interactions -- beyond the literal meaning of the words.  In this talk, we describe both computational models for multi-domain disfluency detection and analyses of different corpora that provide insights into what disfluencies can tell us about the speaker in both high-stakes and casual contexts.

Mari Ostendorf, an alumna of the Stanford Signal Compression and Classification Group, joined the University of Washington in September 1999.  Previously, she was in the Speech Signal Processing Group at BBN Laboratories (1985-1986), and at Boston University on the faculty of the Electrical and Computer Engineering Department (1987-1999).  In 1995, she was a visiting researcher at the ATR Interpreting Telecommunications Laboratory in Japan, and in 2005-2006, she is a Visiting Professor at the University of Karlsruhe. She teaches undergraduate courses in circuits and signals and systems, and graduate courses on various topics related to statistical signal processing.  Professor Ostendorf is a fellow of IEEE and a member of SWE, ASA and Sigma Xi. She has served on numerous technical and advisory committees.

Prof. Ostendorf's research interests include data compression and statistical pattern recognition, particularly in speech processing applications.  Her recent work includes segment-based acoustic modeling for spontaneous speech recognition, dynamic pronunciation modeling dependence modeling for adaptation, use of out-of-domain data and discourse structure in language modeling, and stochastic models of prosody for both recognition and synthesis.  She has published over 200 papers on various problems in speech and language processing.  She works in the Signal, Speech and Language Interpretation Laboratory, where both undergraduate and graduate students are involved in a variety of research projects related to these problems.

Faculty Host: Carolyn Rose
Instructor: Alex Hauptmann

Many modern IR systems and data exhibit new characteristics which are largely ignored by conventional techniques.  What is missing is an ability for the model to change over time and be responsive to stimulus.  Documents, relevance, users and tasks all exhibit dynamic behavior that is captured in big data sets (typically collected over long time spans) and models need to respond to these changes.  This talk provides an up-to-date introduction to Dynamic Information Retrieval Modeling.  In particular, I will talk about how we model information seeking as a partially observable Markov decision process and achieve high accuracy in the TREC Session Tracks. I will also talk about evaluation in dynamic IR and the TREC Dynamic Domain Track.

Grace Hui Yang is an Assistant Professor in the Department of Computer Science at Georgetown University.  Grace obtained her Ph.D. from the Language Technologies Institute, Carnegie Mellon University in 2011.  Grace's current research interests include dynamic search, search engine evaluation, privacy-preserving information retrieval, and information organization.  Prior to this, she conducted research on question answering, ontology construction, near-duplicate detection, multimedia information retrieval and opinion and sentiment detection.  Grace is a recipient of the National Science Foundation (NSF) Faculty Early Career Development Program (CAREER) Award.  Grace co-chaired the SIGIR 2013-2014 Doctoral Consortium, SIGIR 2017 Workshop, and WSDM 2017 Workshop.  She served as an area chair for SIGIR 2014-2016 and ACL 2016.  Grace also co-organized the TREC 2015-now Dynamic Domain Track

Faculty Host: Jamie Callan
Instructor: Alex Hauptmann

The LTI Student Research Symposium (SRS) is a one-day series of talks and poster presentations designed both to increase awareness of the diverse aspects of language technologies research conducted by students within the LTI, as well as to introduce incoming LTI students to the work of current students.

One of the talks will be selected for a Best Presentation Award, which will be announced at the end of the Symposium. The winner of the Best Presentation Award will receive a cash prize of $500.  Additionally, two Honorable Mentions will be selected, each receiving a cash prize of $100, and two $100 for Best Poster Award will also be presented.


Subscribe to LTI