William W. Cohen

William W. Cohen

Professor, Machine Learning Department and Language Technology Institute, Carnegie Mellon University

Director of the Undergraduate Minor in Machine Learning and Co-Director of the Master's of Science in Machine Learning program

Biography

William Cohen received his bachelor's degree in Computer Science from Duke University in 1984, and a PhD in Computer Science from Rutgers University in 1990. From 1990 to 2000 Dr. Cohen worked at AT&T Bell Labs and later AT&T Labs-Research, and from April 2000 to May 2002 Dr. Cohen worked at Whizbang Labs, a company specializing in extracting information from the web. Dr. Cohen is a past president of the International Machine Learning Society, an Action Editor for the Journal of Machine Learning Research, and an Action Editor for the journal ACM Transactions on Knowledge Discovery from Data. He is also an editor, with Ron Brachman, of the AI and Machine Learning series of books published by Morgan Claypool. In the past he has also served as an action editor for the journal Machine Learning, the journal Artificial Intelligence, and the Journal of Artificial Intelligence Research. He was General Chair for the 2008 International Machine Learning Conference, held July 6-9 at the University of Helsinki, in Finland; Program Co-Chair of the 2006 International Machine Learning Conference; and Co-Chair of the 1994 International Machine Learning Conference. Dr. Cohen was also the co-Chair for the 3rd Int'l AAAI Conference on Weblogs and Social Media, which was held May 17-20, 2009 in San Jose, and was the co-Program Chair for the 4rd Int'l AAAI Conference on Weblogs and Social Media. He is a AAAI Fellow, and was a winner of the 2008 the SIGMOD "Test of Time" Award for the most influential SIGMOD paper of 1998, and the 2014 SIGIR "Test of Time" Award for for the most influential SIGIR paper of 2002-2004.

Dr. Cohen's research interests include information integration and machine learning, particularly information extraction, text categorization and learning from large datasets. He holds seven patents related to learning, discovery, information retrieval, and data integration, and is the author of more than 200 publications.

Projects

Projects I'm currently (or recently) involved with include:

NELL, a large-scale information extraction system (which uses SEAL, as well as other techniques).
SimStudent, a project that adds learning-by-demonstration to CTAT.
Minorthird, an open-source Java package of information extraction software. (Note: we've migrated the code now from SourceForge to GitHub.)

Software and demos

Demos:

Here are two movies that demo SimStudent, a programming-by-demonstration system for constructing cognitive tutors, built by Noboru Matsuda.
- Interactive mode (solves problems proactively, as way of posing queries)
- Non-interactive mode

Software:

Ramnath Balasubramanyan's BlockLDA code, as well as some of the other algorithms from his thesis, is available on GitHub.
Code for Ni Lao's PRA method (described in our ECML paper) is now available.
Frank Lin's home page now contains
- the code for power iteration clustering (the algorithm described in our ICML-2010 paper) as well as the datasets we used in the experiments.
- the code for MultiRandomWalk (the semi-supervised learning algorithm described in our ASONAM-2010 paper) as well as the datasets we used in those experiments.
Minorthird is an open-source Java package of information extraction and text classification learning tools.
- I am now distributing a standalone tool, built on Minorthird, for annotating biomedical text. This is particularly aimed at annotating figure captions but might be useful for other text as well. The jar file for this is rather large (17M), as it includes a Minorthird jar. There is documentation available for this, and some sample data.
- My former student Vitor Carvalho distributes the poetically named Jangada and Ciranda, which are also standalone apps built on top of Minorthird, to analyze email messages.
SecondString is another open-source Java package, of approximate string matching techniques.
- SecondString includes a jar for part of an ancient version of Minorthird. For those that are interested in the source behind the mysterious cls.jar, here it is.
SLIPPER is an old old rule-learning system Yoram Singer and I developed. This code is provided with absolutely no warranty, promise of support, or really, any expectation that it will keep working. You are totally on your own with this one, friend.
WHIRL is another old system I wrote. Currently, I am not distributing it, but ask me if you're interested in reviving the source code.
To get a copy of RIPPER, please send mail to my evil twin brother, wcohen -AT- gmail.com. As an alternative to that ancient code: I haven't used it myself, but I've heard good things about J-RIP, a Ripper clone written for WEKA.

Datasets

The following datasets are available for anyone to use for research purposes:

Coordinate terms extracted from a MALT-parsed corpus with 230B sentences, produced by Malcolm Greaves. (Corpus is ClueWeb 2009, Wikipedia from November 2011, Project Gutenberg, and Citeseer.)
Data sets for my paper "Crowdsourced Comprehension: Predicting Prerequisite Structure in Wikipedia" with Partha Talukdar from BEA-2012.
Collections of HTML Tables, hyponyms, as well as extracted entity clusters and MLT evaluations, all associated with Bhavana Dalvi's paper on WebSets from WSDM-2012.
The network datasets used in the experiments of our ICML-2010 paper are on Frank Lin's home page.
100,000+ bibliography entries, in the original BibTeX format, converted to an EndNote-like format, and in a featurized format, for experiments with matching (60M).
A 56k-node, 200k-edge graph containing data from SGD and PubMed, used in Querendipity.
617 messages from 20 Newsgroups, annotated for reply bodies and signatures, prepared by my former student Vitor Carvalho
Two subsets of the Enron data, annotated with person names, prepared by my student Einat Minkov.
Enron email dataset (400Mb, once you get there) contains 800,000+ emails from 150 users+ organized into 4700+ folders.
Some more email data: about two thousand messages released to the public as part of the ongoing investigation of US Attorney firings at the Dept of Justice. This is very strange data---the original email is released as scanned printouts in PDF (?!), so most of the text is not available. There are links to copies of the PDF, some manually added annotations, and a (apparently manually-reconstructed) social network graph. About 1.5Mb (in Excel format). From Mark Johnson, and a network of volunteers.
A collection of various extraction datasets in Minorthird format (6Mb), including about 1000 Enron emails tagged for person names and temporal expressions.
classify.tar.gz (0.4Mb) contains nine problems in which the goal is to classify short entity names. This data was used in Joins that Generalize: Text Classification Using WHIRL (KDD-98).
ranking.tar.gz (8Mb) contains the data used for the meta-search experiments in my JAIR paper Learning to Order Things (with Rob Schapire and Yoram Singer).
match.tar.gz (0.7Mb) contains a suite of labeled entity-name matching and clustering problems (i.e. problems for which the correct matches/clusters are provided), in a single consistent format. In most cases WHIRL's performance is given as a benchmark. (These are also distributed in the RIDDLE Repository. Extraction-oriented versions of some of this data are available on the RISE Repository. (I.e., represented as a problem of extracting data from a website, rather than matching two datasets).)
whirl-bench.tgz (1.1Mb) contains some more WHIRL-format entity name matching problems.

Talks and presentations

Learning to Reason with Extracted Information, keynote talk given March 2014 at Google's Natural Language Understanding Workshop, Zurich, Switzerland.
Learning to Construct and Reason with a Large KB of Extracted Information, invited talk given August 2013 at the Inductive Logic Programming Conference, in Rio de Janeiro, Brazil.
Reasoning With Data Extracted from The Biomedical Literature, invited talk at a joint session of the AAAI Fall Symposia on Discovery Informatics, and Information Retrieval and Knowledge Discovery in Biomedical Text.
Learning Similarity Relations Based on Random Walks in Graphs, invited talk at CIKM 2012, October, 2012.
- Earlier version of talk:Learning Relationships Defined by Linear Combinations of Constrained Random Walks, invited talk at the 9th Workshop on Machine Learning and Graphs, San Diego, CA, Aug 2011.
Fast Effective Clustering for Graphs and Documents, given at CMU's LTI Colloquium Feb 10, 2012.
- Earlier versions given at Virginia Tech in April 2010 and University of Pennsylvania in Feb 2010.
Learning to Extract a Broad-Coverage Knowledge Base from the Web, invited talk at the Symposium on Data-Intensive Analysis, Analytics, and Informatics, Pittsburgh, PA Apr 2011.
Open Information Extraction Methods: Computers that Learn to Read, invited talk at National Federation of Advanced Information Services (NFAIS), Philadelpha, PA, Feb 2011.
Learning Proximity Relations Defined by Linear Combinations of Constrained Random Walks, given at a seminar at the University of Maryland in Sep 2010.
Modeling Entity-Entity Links and Entity-Annotated Text, given at the ICML 2010 Workshop on Topic Modeling.
Predictively Modeling Social Media, invited talk given at the 1st International Workshop on Mining Social Media, co-located with 13th Conference of the Spanish Association for Artificial Intelligence (CAEPIA-TTIA 2009).
Matching and clustering product descriptions using learned similarity metrics, invited talk given at the IJCAI 2009 Workshop on Information Integration on the Web, July 2009. (Powerpoint; 6.7M)
Open information extraction talks:
- Graph-Based Methods for Open Information Extraction, talk given at Nov 2009 at MAVIR in Madrid, Spain.
- Graph-Based Methods for Open Information Extraction, earlier version of talk given at Stanford and Google March 2009.
- Graph-Based Methods for Open Information Extraction, still earlier version of the same talk given at a 2008 NIPS workshop.
- A QT video of highlights from the workshop talks, including an incisive technical question addressed to me from my colleague Steve Fienberg.
Embodied Cognition and Knowledge: Integration of Heterogeneous Databases without Common Domains Using Queries Based on Textual Similarity, talk given for my 10-year "Test of Time" Award at SIGMOD-2008(Powerpoint; 11Mb)
Using Machine Learning to Discover and Understand Structured Data, invited talk given at LinkedData 2008. (Powerpoint; 6Mb)
Machine Learning for Personal Information Management, invited talk given at ICMLA-2007. (Powerpoint; 8Mb)
A Framework for Learning to Query Heterogeneous Data, invited talk given at IQIS 2006. (Powerpoint; 8Mb)
On Beyond Hypertext: Searching in Graphs Containing Documents, Words, and Actual Data, invited talk given at DB/IR Day 2006. (Powerpoint; 6Mb)
A Century Of Progress On Information Integration: A Mid-Term Report, an overview of information integration, focusing modestly on my own work, given as invited talk at WebDB-2005. (Powerpoint; 12Mb)

Tutorials:
- Information extraction (PowerPoint; 4.8Mb), aimed at folks somewhat familiar with statistical NLP methods. And thanks to Thierry Poibeau, there's also a version en francais (did I get that right, Thierry?) Also, two earlier versions of this are also still around, both given with Andew McCallum at recent conferences, KDD-2003(PowerPoint; 6.8Mb) and NIPS-2002.
- Text classification (PowerPoint; 3Mb), given at a CALD Summer Course.
- Collaborative filtering (PowerPoint; 9.1Mb), given at a DIMACS workshop.
A mini-course on record linkage and matching:
- Overview of record linkage methods(PowerPoint; 250kb).
- Overview of distance metrics for strings(PowerPoint; 530kb).
- Overview of using HMMs for normalizing text in record linkage tasks(PowerPoint; 640kb).
  It's not a presentation, but I have also put together a short annotated bibliography of record linkage and matching papers.
- William Hayes has a nice summary of an extended discussion of string-matching tools on the BioNLP mailing list (July 2012).
Other technical talks:
- A presentation of my IJCAI-2005 results on "stacked sequential learning", presented in Edinburgh in August, 2005.
- A presentation of my NIPS-2002 results on using bootstrapping techniques to improve web page classification, given at CMU in October 2002. (PowerPoint; 3.2mb).
- A presentation of my WWW-2002 results on wrapper learning, presented in April 2002. (PDF; 170kb).
- An overview of experiments with WHIRL. (PDF; 800kb).

Teaching

Current/upcoming:

Fall 2014: 10-601 Machine Learning, Tu-Thu 1:30-2:50, Wean 7500

Past courses:

Spring 2014: 10-605 Machine Learning with Large Datasets, Mon-Wed 1:30-2:50, Dougherty Hall 1112
Fall 2013: 10-601 Machine Learning, Mon-Wed 4:30-5:50, Doherty Hall 2315 (with Eric Xing).
Spring 2013: Machine Learning with Large Datasets, Mon-Wed 1:30-2:50, 4307 GHC
Fall 2012: ML 10-802 and LTI 11-772 (Analysis of Social Media), 10:30-11:50pm Tues & Thus, 4303 Gates Building.
Fall 2012: 10-915, the MLD Journal Club, 12-1:20pm Tue & Thu, 4101 Gates Building (with Roy Maxion).
Spring 2012: Machine Learning with Large Datasets, Tues-Thurs 1:30-2:50pm, NSH 1305
Fall 2011: Structured Prediction for Language and Other Discrete Data (SPLODD-2011), ML 10-710 and LTI 11-763, Tues-Thursday 3:00-4:20 in Gates-Hillman 4211. This is co-taught by myself and Noah Smith, and will include some subjects from Information Extraction and some from Language and Stats 2. A machine learning course (10-701 or consent of the instructors) is a prereq; we don't recommend that you take the course if you have already taken Information Extraction or Language and Stats 2.
Spring 2011: ML 10-802 and LTI 11-772 (Analysis of Social Media), 10:30-11:50pm Tues & Thus, 4303 Gates Building.
Spring 2011: 10-915, the MLD Journal Club, 3-4pm Mon & Wed, 4101 Gates Building.
Fall 2010: 10-707 (Information Extraction - cross-listed in LTI as 11-748), 1:30-2:50pm Mon & Wed, Gates 4101. The first class is 9/8, the Wed after Labor Day, to allow incoming students time to attend the IC courses.
Spring 2010: 10-802 (Analysis of Social Media).
Fall 2009: 10-707 (Information Extraction), 1:30-2:50pm Mon & Wed, 5222 Gates Building.
Spring 2008: 10-601 (Machine Learning) with Tom Mitchell, on 3-4:30 Mon & Wed in Wean Hall 5409.
Fall 2007: Analysis of Social Media, Machine Learning 10-802 and LTI 11-772, with Natalie Glance (of Google Pittsburgh) - a brand-new seminar course. 4:30-6:30 Tuesdays in Wean Hall 4623.
- Note: This site is the shattered remains of a once-beautiful wiki, created by the students of 10-802, generously hosted for free by ScribbleWiki, tragically lost (due a combination of RAID drive failures and low-bidder backup schemes), and then largely recovered using Warrick from various internel caches and archives.
Fall 2007: Current Topics in Computational Biology (Journal Club), 02-701. (Announcements). Thursdays from 4:00-5:00 in 411 Mellon Institute (after Cell & Systems Modeling).
Spring 2007: Information Extraction, Machine Learning 10-707 and LTI 11-748 - back by popular demand for the first time since 2004!
Fall 2006: Current Topics in Computational Biology (Journal Club), 02-701. (Announcements)
Spring 2006: Read the Web, CALD 10-709.
June 21,23,25, 2005: A mini-course on Minorthird. Materials are below.
- Slides, notes, and sample files from first day's lecture.
- Slides, notes, and sample files from second day's lecture.
- Powerpoint slides from third day's lecture.
- Jar file for minorThird, if you only want to run the code, not compile it or read it. The installation process here is:
  1. Install Java 1.4 or higher (actually, JRE is all you need).
  2. Download the jar for minorThird and stick it in some directory.
  3. Optionally, download the sample data repository and unpack it into the same directory.
  4. Change to that same directory and then run Minorthird with the command
    java -Xmx500M -jar minorthird.jar
    What will pop up will be a small launch pad that can be used to start any of the UI programs. You can also start a particular main by specifying minorthird.jar as your classpath, for instance:
    java -Xmx500M -cp minorthird.jar edu.cmu.minorthird.ui.Help
- If you want to do a real install here's the home page on Sourceforge, and a document on how to do a CVS install Minorthird.
Spring 2004: "Learning to Turn Words into Data: Machine Learning Approaches to Information Extraction and Information Integration", CALD 10-707 and LTI 11-748.

Publications

Here's an RSS feed of my papers. (Note: the feed I had created with Dapper seems spam-infested now.) Here's a pointer to my DBLP page.
A Computer Scientist's Guide To Biology is no longer available from this web page, but is now available from Springer. Here is a the TOC, introduction, index, and a sample chapter, from a late draft of the book; and also all the figures from the book in PowerPoint and all the figures in PDF. (The figures are a little prettier than the ones in the final book, which is black and white, not color).
ICML 2006 Proceedings are available in print, for the true afficianado of fine learning-related research. It's well worth the money for the cover art alone (of course, all the papers are also available on-line for free.)
Recent and selected publications. These are some representative publications for which on-line copies can be distributed.
All publications. Here is an more-or-less complete chronological list of my publications. The bibliography includes pointers to on-line versions when I can provide them, but unfortunately copyright restrictions don't allow me to make all of my publications available on-line. Of course, reprints are always available from me on request.
Publications by topic:

Recent papers I'm keeping in HTML or PDF (which requires Adobe Acrobat Reader to view). Older papers are mostly in Postscript. For Windows, I use the GSView reader for postscript. Most of these papers are viewable in several formats in ResearchIndex.

Students and other colleagues

William Yang Wang, LTI PhD student
Dana Movshovitz-Attias, CSD PhD student.
Bhavana Dalvi Mishra, LTI PhD student (co-advised with Jamie Callan)
Rahul Goutam, MLT MLT student, co-advised with
Yu-Hsin Allen Kuo, MLT MLT student, co-advised with Natasa Miskov-Zivanov
Katie Rivard Mazaitis, research programmer/analyst
Malcolm Greaves, former CSD master's student.
Tae Yano, (former LTI PhD student, co-advised with Noah Smith, now at Microsoft)
Nan Li, (former CSD PhD student, co-advised with Ken Koedinger, now at code.org)
Ramnath Balasubramanyan, (LTI PhD student, now at Twitter)
Mahesh Joshi, (former LTI PhD student, co-advised with Carolyn Rosé, now at EBay)
Frank Lin, (former LTI PhD student, now at Twitter)
Ni Lao (former LTI PhD student, now at Google)
Richard C. Wang, (former LTI PhD student co-advised with Bob Frederking, now at Enfind).
Andrew Arnold (former MLD PhD student, now at Ophir Partners LLC)
Einat Minkov (former LTI PhD student, now at Haifa University)
Vitor Rocha de Carvalho (former LTI PhD student, now at QualComm)
Zhenzhen Kou (former MLD PhD student, now at Google)
Edoardo Airoldi (former MLD/Stats PhD student, co-advised with Steve Fienberg)
Ja-Hui Chang (visiting faculty from National Central University, Taiwan, 2007-2008)
Tuan Ahn Hoang, (PhD student at Singapore Management University, visited CMU for 2012-2013 academic year in my group).
Freddy Chong Tat Chua (PhD student at Singapore Management University, visited CMU for the academic year 2011-2012 in my group.)
Gustavo Lacerda (former research assistant, co-supervised with Noboru Matsuda and Ken Koedinger, now at UBC)
Ramesh Nallapati (former postdoc, co-supervised with John Lafferty, now at IBM Watson)
Noboru Matsuda (former postdoc, co-supervised with Ken Koedinger, now System Scientist in CMU's HCII)
Pradeep Ravikumar (former MLD PhD student, co-advised with Steve Fienberg)
I have been an external committee member for the PhD theses of
- John Zelle (degree from U Texas)
- Misha Bilenko (from U Texas)
- Daniel Kudenko (Rutgers)
- Chumki Basu (Rutgers)
- Ananlada Chotimongkol (CMU)
- Wei-Hao Lin (CMU)
- Cenk Gazen (CMU)
- David Nadeau (U Ottowa)
- Hanghang Tong (CMU)
- Ben van Durme (Rochester)
- Partha Talukdar (U Penn)
- Andy Carlson (CMU)
- Yifen Huang (CMU)
- Swapna Sundaran (U Pitt)
- Michael Heilman (CMU)
- Jon Elsas (CMU)
- Dipanjan Das (CMU)
- Fan Guo (CMU)
- Jana Diesner (CMU)
- Freddy Chong Tat Chua (Singapore Management University).
I have also been an external committee member for the Master's theses of Mehrbod Sharifi (CMU) and Weam Abu-Zaki (CMU).
I am currently an external committee member for Jay Pujara (Univ Maryland), and Steven Gardiner, Justin Betteridge, Qirong Ho, Dani Koutra, Reyyan Yeniterzi, and YiChia Wang (all at CMU).
Contact Info

William W. Cohen
Professor, Machine Learning Department
Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213
8217 Gates Hillman Complex
(shipping address: 6105 Gates Hillman Complex)
voice: 412-268-7664 / fax: 412-268-2205
Assistant: Sandy Winkler, sandyw@cs.cmu.edu, 412-268-xxxx

My preferred email address is: wcohen AT cs DOT cmu DOT edu
Other Stuff

For those many friends whose research I have built on, be warned. My full name, "William Weston Cohen", is an anagram of the phrase "I now cite shallow men". (From Sara Cohen - no relation! - comes this warning: "Women's rights activists would probably request you to use the following anagram instead: 'I shall now cite women'".)
I am often praised for my highly artistic and functional web site designs. An example is the site for SC Indexing, a professional book indexer. However, I accept few clients - this one happens to be my wife.
Through my advisor, Alex Borgida, I can trace my "academic lineage" back to luminaries like Leibniz, Newton and Alfred Whitehead.
In 2014 I unearthed a strange relic from the past, a sort of game/website I wrote for my son Charlie back in...I'm gonna say, 1994, 1995, something like that, and I sort of made it work again, although JavaScript has changed a bit in the last couple of decades. (The main bugs have to do with sound-file presentation - in 1994 these were played by mime-file configured helper programs, not natively by the browser, so now you need to hit 'back' about 1/2 the time after a sound plays.) Historically interesting? You decide!
When I'm not working my day job, I avoid productive behavior by playing music.
Poetry anyone?