William W. Cohen Professor, Machine Learning Department and Language Technology Institute, Carnegie Mellon University Director of the Undergraduate Minor in Machine Learning and Co-Director of the Master of Science in Machine Learning program

Biography

Dr. Cohen's research interests include information integration and machine learning, particularly information extraction, text categorization and learning from large datasets. He has a long-standing interest in statistical relational learning and learning models, or learning from data, that display non-trivial structure. He holds seven patents related to learning, discovery, information retrieval, and data integration, and is the author of more than 200 publications.

Announcements and FAQs

• Can I visit CMU and work with you? Prospective visitors and colleagues, my apologies for not answering all emails of inquiry personally. Right now there is a major space crunch in CMU, and I'm also just about at the limit of my (rather limited) managerial abilities. Except in an unusual circumstance (like existing collaborations) I will not be able to host any long-term visitors to my group, even if you're self-funded. Again, except in unusual circumstances, I will not supervise students remotely for a research project.

• Can I apply to CMU and work with you as a grad student? Prospective students, again, my apologies for not answering all emails of inquiry personally. At CMU graduate admissions is independent of an advisor's preferences, but instead is based on the strength of the student's record relative to the other applicants. So, if you're interested in working with me, the first step is to apply and be accepted. After that, you're welcome to contact me. Normally I make decisions during the usual "matching period" in the fall, which I believe to be the best timeline for the students.

• What's the difference between 10-601 and ...? If you're having trouble with the MLD's growing menu of intro ML courses here's a draft of a document that explains the differences. If you're not sure if you're qualified, the prereqs for the course are listed on the course home page, and we're fairly strict about enforcing them for undergrads. Grad students should have equivalent experience: good programming skills - the equivalent of a one-semester college course - and some mathematical maturity, including prior exposure to calculus, probability and linear algebra. If you're not sure about your background there is a self-assessment test you can take.

Projects

• GNAT is an automatic KB construction toolkit that has been used to build KBs for several different domains, including consumer health information and software.
• NELL is a web-scale information extraction system.

Software and demos

• ProPPR is a "locally groundable" probabilistic first-order logic.
• TensorLog is a newer probabilistic first-order logic which is fully differentiable.
• Guinea Pig is a pure Python workflow language for Hadoop.

• Bhuwan Dhingra is distributing an updated version of the Gated Attention Reader via Github. As of Dec 2016 the GA Reader is obtaining state-of-the-art results on several of the standard benchmarks for answering cloze questions.
• Here is a comment-completion Plugin for Eclipse, from Dana Movshovitz-Attias.
• Here is Ramnath Balasubramanyan's BlockLDA code, as well as some of the other algorithms from his thesis, is available on GitHub.
• Code for Ni Lao's PRA method (described in our ECML paper) is available.
• Frank Lin's home page contains
  • the code for power iteration clustering (the algorithm described in our ICML-2010 paper) as well as the datasets we used in the experiments.
  • the code for MultiRandomWalk (the semi-supervised learning algorithm described in our ASONAM-2010 paper) as well as the datasets we used in those experiments.

• Minorthird is an open-source Java package of information extraction and text classification learning tools. This package is stable but not being actively maintained.
  • I there is also a standalone tool, built on Minorthird, for annotating biomedical text. This is particularly aimed at annotating figure captions but might be useful for other text as well. The jar file for this is rather large (17M), as it includes a Minorthird jar. There is documentation available for this, and some sample data.
  • My former student Vitor Carvalho distributes the poetically named Jangada and Ciranda, which are also standalone apps built on top of Minorthird, to analyze email messages.
• SecondString is another open-source Java package, of approximate string matching techniques.
  • SecondString includes a jar for part of an ancient version of Minorthird. For those that are interested in the source behind the mysterious cls.jar, here it is.
• SLIPPER is an old old rule-learning system Yoram Singer and I developed. This code is provided with absolutely no warranty, promise of support, or really, any expectation that it will keep working. You are totally on your own with this one, friend.
• WHIRL is another old system I wrote. Currently, I am not distributing it, but ask me if you're interested in reviving the source code.
• To get a copy of RIPPER, please send mail to my evil twin brother, wcohen -AT- gmail.com. As an alternative to that ancient code: I haven't used it myself, but I've heard good things about J-RIP, a Ripper clone written for WEKA.

Datasets

• Zhilin Yang is distributing the data from our ACL-2017 paper on semi-supervised QA.
• Lidong Bing has distributed two datasets from our joint work: the data used in our EMNLP 2015 paper, Improving Distant Supervision for Information Extraction Using Label Propagation Through List, and also the dataset used in our AAAI 2016 paper, Distant IE by Bootstrapping Using Lists and Document Structure. The data extracted by this system can also be browsed.
• Ni Lao has distributed the labeled data from our EMNLP 2010 paper, Random Walk Inference and Learning in A Large Scale Knowledge Base, both Turker-labeled data and NELL pseudo labels.
• Coordinate terms extracted from a MALT-parsed corpus with 230B sentences, produced by Malcolm Greaves. (Corpus is ClueWeb 2009, Wikipedia from November 2011, Project Gutenberg, and Citeseer.)
• Data sets for my paper "Crowdsourced Comprehension: Predicting Prerequisite Structure in Wikipedia" with Partha Talukdar from BEA-2012.
• Collections of HTML Tables, hyponyms, as well as extracted entity clusters and MLT evaluations, all associated with Bhavana Dalvi's paper on WebSets from WSDM-2012.
• The network datasets used in the experiments of our ICML-2010 paper are on Frank Lin's home page.
• 100,000+ bibliography entries, in the original BibTeX format, converted to an EndNote-like format, and in a featurized format, for experiments with matching (60M).
• A 56k-node, 200k-edge graph containing data from SGD and PubMed, used in Querendipity.
• 617 messages from 20 Newsgroups, annotated for reply bodies and signatures, prepared by my former student Vitor Carvalho
• Two subsets of the Enron data, annotated with person names, prepared by my student Einat Minkov.
• Enron email dataset (400Mb, once you get there) contains 800,000+ emails from 150 users+ organized into 4700+ folders.
• Some more email data: about two thousand messages released to the public as part of the ongoing investigation of US Attorney firings at the Dept of Justice. This is very strange data---the original email is released as scanned printouts in PDF (?!), so most of the text is not available. There are links to copies of the PDF, some manually added annotations, and a (apparently manually-reconstructed) social network graph. About 1.5Mb (in Excel format). From Mark Johnson, and a network of volunteers.
• A collection of various extraction datasets in Minorthird format (6Mb), including about 1000 Enron emails tagged for person names and temporal expressions.
• classify.tar.gz (0.4Mb) contains nine problems in which the goal is to classify short entity names. This data was used in Joins that Generalize: Text Classification Using WHIRL (KDD-98).
• ranking.tar.gz (8Mb) contains the data used for the meta-search experiments in my JAIR paper Learning to Order Things (with Rob Schapire and Yoram Singer).
• match.tar.gz (0.7Mb) contains a suite of labeled entity-name matching and clustering problems (i.e. problems for which the correct matches/clusters are provided), in a single consistent format. In most cases WHIRL's performance is given as a benchmark. (These are also distributed in the RIDDLE Repository. Extraction-oriented versions of some of this data are available on the RISE Repository. (I.e., represented as a problem of extracting data from a website, rather than matching two datasets).)
• whirl-bench.tgz (1.1Mb) contains some more WHIRL-format entity name matching problems.

Talks and presentations

• An invited talk given at SNL-2017 (the 1st International Workshop on Symbolic-Neural Learning) in July 2017.
• An invited talk given at WAKBC-2016 in June 2016.
• Tutorial on statistical relational learning given at NAACL 2016 with William Wang (a shorter version of this was also presented at IJCAI 2016):
  • Part 1 - overview on logic, probability, MLNs, and probabilistic DDBs
  • Part 2 - ProPPR and applications
  • Part 3 - TensorLog, and other recent and current work
• Series of three lectures on probabilistic logic programs given at Singapore Management University in Feb 2016:
  • Background on logic and probabilistic models
  • Parameter learning and structure learning in ProPPR
  • Joint learning in ProPPR and comparing to neural approaches
• Can KR Represent Real-World Knowledge?, invited talk given March 2015 at the AAAI Spring Symposium on Knowledge Representation and Reasoning: Integrating Symbolic and Neural Approaches
• Learning to Reason with Extracted Information, keynote talk given March 2014 at Google's Natural Language Understanding Workshop, Zurich, Switzerland.
• Learning to Construct and Reason with a Large KB of Extracted Information, invited talk given August 2013 at the Inductive Logic Programming Conference, in Rio de Janeiro, Brazil.
• Reasoning With Data Extracted from The Biomedical Literature, invited talk at a joint session of the AAAI Fall Symposia on Discovery Informatics, and Information Retrieval and Knowledge Discovery in Biomedical Text.
• Learning Similarity Relations Based on Random Walks in Graphs, invited talk at CIKM 2012, October, 2012.
  • Earlier version of talk:Learning Relationships Defined by Linear Combinations of Constrained Random Walks, invited talk at the 9th Workshop on Machine Learning and Graphs, San Diego, CA, Aug 2011.
• Fast Effective Clustering for Graphs and Documents, given at CMU's LTI Colloquium Feb 10, 2012.
  • Earlier versions given at Virginia Tech in April 2010 and University of Pennsylvania in Feb 2010.
• Learning to Extract a Broad-Coverage Knowledge Base from the Web, invited talk at the Symposium on Data-Intensive Analysis, Analytics, and Informatics, Pittsburgh, PA Apr 2011.
• Open Information Extraction Methods: Computers that Learn to Read, invited talk at National Federation of Advanced Information Services (NFAIS), Philadelpha, PA, Feb 2011.
• Learning Proximity Relations Defined by Linear Combinations of Constrained Random Walks, given at a seminar at the University of Maryland in Sep 2010.
• Modeling Entity-Entity Links and Entity-Annotated Text, given at the ICML 2010 Workshop on Topic Modeling.
• Predictively Modeling Social Media, invited talk given at the 1st International Workshop on Mining Social Media, co-located with 13th Conference of the Spanish Association for Artificial Intelligence (CAEPIA-TTIA 2009).
• Matching and clustering product descriptions using learned similarity metrics, invited talk given at the IJCAI 2009 Workshop on Information Integration on the Web, July 2009. (Powerpoint; 6.7M)
• Open information extraction talks:
  • Graph-Based Methods for Open Information Extraction, talk given at Nov 2009 at MAVIR in Madrid, Spain.
  • Graph-Based Methods for Open Information Extraction, earlier version of talk given at Stanford and Google March 2009.
  • Graph-Based Methods for Open Information Extraction, still earlier version of the same talk given at a 2008 NIPS workshop.
  • A QT video of highlights from the workshop talks, including an incisive technical question addressed to me from my colleague Steve Fienberg.
• Embodied Cognition and Knowledge: Integration of Heterogeneous Databases without Common Domains Using Queries Based on Textual Similarity, talk given for my 10-year "Test of Time" Award at SIGMOD-2008(Powerpoint; 11Mb)
• Using Machine Learning to Discover and Understand Structured Data, invited talk given at LinkedData 2008. (Powerpoint; 6Mb)
• Machine Learning for Personal Information Management, invited talk given at ICMLA-2007. (Powerpoint; 8Mb)
• A Framework for Learning to Query Heterogeneous Data, invited talk given at IQIS 2006. (Powerpoint; 8Mb)
• On Beyond Hypertext: Searching in Graphs Containing Documents, Words, and Actual Data, invited talk given at DB/IR Day 2006. (Powerpoint; 6Mb)
• A Century Of Progress On Information Integration: A Mid-Term Report, an overview of information integration, focusing modestly on my own work, given as invited talk at WebDB-2005. (Powerpoint; 12Mb)
• Tutorials:
  • Information extraction (PowerPoint; 4.8Mb), aimed at folks somewhat familiar with statistical NLP methods. And thanks to Thierry Poibeau, there's also a version en francais (did I get that right, Thierry?) Also, two earlier versions of this are also still around, both given with Andew McCallum at recent conferences, KDD-2003(PowerPoint; 6.8Mb) and NIPS-2002.
  • Text classification (PowerPoint; 3Mb), given at a CALD Summer Course.
  • Collaborative filtering (PowerPoint; 9.1Mb), given at a DIMACS workshop.

• A mini-course on record linkage and matching:
  • Overview of record linkage methods(PowerPoint; 250kb).
  • Overview of distance metrics for strings(PowerPoint; 530kb).
  • Overview of using HMMs for normalizing text in record linkage tasks(PowerPoint; 640kb).
    It's not a presentation, but I have also put together a short annotated bibliography of record linkage and matching papers.
  • William Hayes has a nice summary of an extended discussion of string-matching tools on the BioNLP mailing list (July 2012).

• Other technical talks:
  • A presentation of my IJCAI-2005 results on "stacked sequential learning", presented in Edinburgh in August, 2005.
  • A presentation of my NIPS-2002 results on using bootstrapping techniques to improve web page classification, given at CMU in October 2002. (PowerPoint; 3.2mb).
  • A presentation of my WWW-2002 results on wrapper learning, presented in April 2002. (PDF; 170kb).
  • An overview of experiments with WHIRL. (PDF; 800kb).

Teaching

• Fall 2017: Machine Learning with Large Datasets, 10-605 and 10-805, Tues-Thus 1:30-2:50pm, room TBA.
• Spring 2018: Undergraduate Level Machine Learning with Large Datasets, 10-405?, time and room TBA.

• Fall 2016: Machine Learning with Large Datasets, 10-605 and 10-805, Tues-Thus 1:30-2:50pm, Wean Hall 7500.
• Spring 2016: Machine Learning 10-601, Mon-Wed time 10:30-11:50am, GHC 4401, with Maria-Florina Balcan.
• Fall 2015: Machine Learning with Large Datasets, 10-605 and 10-805, Tu-Thu 4:30-5:50am in Dougherty Hall 2210.
• Spring 2015: Machine Learning with Large Datasets, 10-605 and 10-805, Tu-Thu 10:30-11:50am in BH A51
• Fall 2014: 10-601 Machine Learning, Tu-Thu 1:30-2:50, Wean 7500
• Spring 2014: 10-605 Machine Learning with Large Datasets, Mon-Wed 1:30-2:50, Dougherty Hall 1112
• Fall 2013: 10-601 Machine Learning, Mon-Wed 4:30-5:50, Doherty Hall 2315 (with Eric Xing).
• Spring 2013: Machine Learning with Large Datasets, Mon-Wed 1:30-2:50, 4307 GHC
• Fall 2012: ML 10-802 and LTI 11-772 (Analysis of Social Media), 10:30-11:50pm Tues & Thus, 4303 Gates Building.
• Fall 2012: 10-915, the MLD Journal Club, 12-1:20pm Tue & Thu, 4101 Gates Building (with Roy Maxion).
• Spring 2012: Machine Learning with Large Datasets, Tues-Thurs 1:30-2:50pm, NSH 1305
• Fall 2011: Structured Prediction for Language and Other Discrete Data (SPLODD-2011), ML 10-710 and LTI 11-763, Tues-Thursday 3:00-4:20 in Gates-Hillman 4211. This is co-taught by myself and Noah Smith, and will include some subjects from Information Extraction and some from Language and Stats 2. A machine learning course (10-701 or consent of the instructors) is a prereq; we don't recommend that you take the course if you have already taken Information Extraction or Language and Stats 2.
• Spring 2011: ML 10-802 and LTI 11-772 (Analysis of Social Media), 10:30-11:50pm Tues & Thus, 4303 Gates Building.
• Spring 2011: 10-915, the MLD Journal Club, 3-4pm Mon & Wed, 4101 Gates Building.
• Fall 2010: 10-707 (Information Extraction - cross-listed in LTI as 11-748), 1:30-2:50pm Mon & Wed, Gates 4101. The first class is 9/8, the Wed after Labor Day, to allow incoming students time to attend the IC courses.
• Spring 2010: 10-802 (Analysis of Social Media).
• Fall 2009: 10-707 (Information Extraction), 1:30-2:50pm Mon & Wed, 5222 Gates Building.
• Spring 2008: 10-601 (Machine Learning) with Tom Mitchell, on 3-4:30 Mon & Wed in Wean Hall 5409.
• Fall 2007: Analysis of Social Media, Machine Learning 10-802 and LTI 11-772, with Natalie Glance (of Google Pittsburgh) - a brand-new seminar course. 4:30-6:30 Tuesdays in Wean Hall 4623.
  • Note: This site is the shattered remains of a once-beautiful wiki, created by the students of 10-802, generously hosted for free by ScribbleWiki, tragically lost (due a combination of RAID drive failures and low-bidder backup schemes), and then largely recovered using Warrick from various internel caches and archives.
• Fall 2007: Current Topics in Computational Biology (Journal Club), 02-701. (Announcements). Thursdays from 4:00-5:00 in 411 Mellon Institute (after Cell & Systems Modeling).
• Spring 2007: Information Extraction, Machine Learning 10-707 and LTI 11-748 - back by popular demand for the first time since 2004!
• Fall 2006: Current Topics in Computational Biology (Journal Club), 02-701. (Announcements)
• Spring 2006: Read the Web, CALD 10-709.
• June 21,23,25, 2005: A mini-course on Minorthird. Materials are below.
  • Slides, notes, and sample files from first day's lecture.
  • Slides, notes, and sample files from second day's lecture.
  • Powerpoint slides from third day's lecture.
  • Jar file for minorThird, if you only want to run the code, not compile it or read it. The installation process here is:
    1. Install Java 1.4 or higher (actually, JRE is all you need).
    2. Download the jar for minorThird and stick it in some directory.
    3. Optionally, download the sample data repository and unpack it into the same directory.
    4. Change to that same directory and then run Minorthird with the command
       java -Xmx500M -jar minorthird.jar

       What will pop up will be a small launch pad that can be used to start any of the UI programs. You can also start a particular main by specifying minorthird.jar as your classpath, for instance:
       java -Xmx500M -cp minorthird.jar edu.cmu.minorthird.ui.Help

  • If you want to do a real install here's the home page on Sourceforge, and a document on how to do a CVS install Minorthird.
• Spring 2004: "Learning to Turn Words into Data: Machine Learning Approaches to Information Extraction and Information Integration", CALD 10-707 and LTI 11-748.

Publications

• Here's an RSS feed of my papers. (Note: the feed I had created with Dapper seems spam-infested now.) Here's a pointer to my DBLP page.
• A Computer Scientist's Guide To Biology is no longer available from this web page, but is now available from Springer. Here is a the TOC, introduction, index, and a sample chapter, from a late draft of the book; and also all the figures from the book in PowerPoint and all the figures in PDF. (The figures are a little prettier than the ones in the final book, which is black and white, not color).
• ICML 2006 Proceedings are available in print, for the true afficianado of fine learning-related research. It's well worth the money for the cover art alone (of course, all the papers are also available on-line for free.)
• Recent and selected publications. These are some representative publications for which on-line copies can be distributed.
• All publications. Here is an more-or-less complete chronological list of my publications. The bibliography includes pointers to on-line versions when I can provide them, but unfortunately copyright restrictions don't allow me to make all of my publications available on-line. Of course, reprints are always available from me on request.
• Publications by topic:
  • Matching/Data Integration
  • Text categorization
  • Information Extraction
  • Rule Learning
  • Collaborative Filtering
  • Applications
  • Formal Results
  • Inductive Logic Programming
  • Explanation-Based Learning

Students and other colleagues