The METEOR Automatic Machine
Translation Evaluation System
Alon Lavie
Carnegie Mellon University
Pittsburgh, PA, USA
Download METEOR
You can download the current version (v0.4.3 - April 18, 2005) of
METEOR from this link.
[View changes between this version and
the previous ones].
About METEOR
METEOR is a system that automatically evaluates the output of machine
translation engines by comparing to them to (one or more) reference
translations. For a given pair of hypothesis and reference strings,
the evaluation proceeds in a sequence of stages, with different
criteria being used at each stage to find and score unigram
matches. By default, at the first stage all exact matches are detected
between the two strings, while in the second stage the words not
matched in the first stage are stemmed using the Porter stemmer and
then matches are found between these stemmed words.
The matching system is written in Perl, and each matching stage
is implemented as a separate Perl module. In addition to the
two default matching modules (exact matching and stemmed matching), a
WordNet based stemmed matching module and a WordNet based synonym
matching module are also provided with this distribution. METEOR can
be run with the default modules, or the user can override the
defaults, and use one or more of the given modules in any order of
preference. Further, the user can write his own matching module and
plug it into the generic matching system.
METEOR's input file format is exactly the same as those of Bleu and
NIST's Machine Translation Evaluation system. Thus all translation
data that can be evaluated using Bleu (such as the TIDES data) can
also be directly evaluated using METEOR.
You can read more details about the current version of METEOR here.
This page last modified on: 18th April, 2005.