Resources

• Einat Amitay's Web IR and IE resource page
• Rainbow Fast C building bag-of-words representations, learning models, and classifying documents - Andrew McCallum Rainbow Documentation
  Local executables:
  • Source: /afs/cs/project/theo-9/webkb/mccallum/src/bow
  • Linux binaries: /afs/cs/project/theo-9/webkb/mccallum/src/bow-linux
  • SUNOS binaries: /afs/cs/project/theo-9/webkb/mccallum/src/bow-sunos
• Wordnet Wordnet home page. This is installed on CS machines - callable as wn - there are manpages installed too.
  • "Five Papers on WordNet" is available in PostScript, as well as other papers on wordnet
  • Jason Rennie's Perl code to interface with Wordnet
• Link Grammar Link Grammar is a simple grammar and parser developed at CMU, with a large coverage vocabulary, and robustness to repetition (but not omission). Dayne is working with the link grammar parser.
• Language Modeling Resources
  • The CMU-Cambridge Statistical Language Modeling Toolkit
    (local installation for Language and Statistics class.)
    This page contains the documentation, as well as links to source etc. Executables are installed in /afs/cs/academic/class/11761-s97/toolkit/CMU-Camb_Toolkit_v2-BETA/bin/ for Intel boxes running NetBSD, Alphas running Dec Unix 3.2, and SunOS. (Master copy at Cambridge University)
  • Class-based n-gram models of natural language by Brown et al -- the paper Andrew Mccallum has given us a tempting fore-taste of.
  • Statistical Languiage Modeling and N-grams - handouts giving an introduction to n-gram-based language modeling, by Roni Rosenfeld for the Language and Statistics class (including descriptions of the perplexity measure and smoothing techniques)
  • The Language and Statistics class home page
• Data Archive Archive of text data sets
• LibParse Dayne's library of text-parsing functions, including html parsing, as well as implementation of perl-like regular expressions in LISP

• Resources on Statistical NLP/corpus-based computational linguistics