---

Inv(FP) Indexer

This application builds an Inv(FP) index for a collection of documents.

To use it, follow the general steps of running a lemur application.

The parameters are:

1. index: name of the index table-of-content file without the .ifp extension.
2. memory: memory (in bytes) of Inv(FP)PushIndex (def = 96000000).
3. position: store position information (def = 1).
4. stopwords: name of file containing the stopword list.
5. acronyms: name of file containing the acronym list.
6. countStopWords: If true, count stopwords in document length.
7. docFormat:
   • "trec" for standard TREC formatted documents
   • "web" for web TREC formatted documents
   • "chinese" for segmented Chinese text (TREC format, GB encoding)
   • "chinesechar" for unsegmented Chinese text (TREC format, GB encoding)
   • "arabic" for Arabic text (TREC format, Windows CP1256 encoding)
8. stemmer:
   • "porter" Porter stemmer.
   • "krovetz" Krovetz stemmer, requires additional parameters
     1. KstemmerDir: Path to directory of data files used by Krovetz's stemmer.
   • "arabic" arabic stemmer, requires additional parameters
     1. arabicStemDir: Path to directory of data files used by the Arabic stemmers.
     2. arabicStemFunc: Which stemming algorithm to apply, one of:
        • arabic_stop : arabic_stop
        • arabic_norm2 : table normalization
        • arabic_norm2_stop : table normalization with stopping
        • arabic_light10 : light9 plus ll prefix
        • arabic_light10_stop : light10 and remove stop words
9. dataFiles: name of file containing list of datafiles to index.

This application builds an Inv(FP) index for a collection of documents.

To use it, follow the general steps of running a lemur application.

The parameters are:

1. index: name of the index table-of-content file without the .ifp extension.
2. memory: memory (in bytes) of Inv(FP)PushIndex (def = 96000000).
3. position: store position information (def = 1).
4. stopwords: name of file containing the stopword list.
5. acronyms: name of file containing the acronym list.
6. countStopWords: If true, count stopwords in document length.
7. docFormat:
   • "trec" for standard TREC formatted documents
   • "web" for web TREC formatted documents
   • "chinese" for segmented Chinese text (TREC format, GB encoding)
   • "chinesechar" for unsegmented Chinese text (TREC format, GB encoding)
   • "arabic" for Arabic text (TREC format, Windows CP1256 encoding)
8. stemmer:
   • "porter" Porter stemmer.
   • "krovetz" Krovetz stemmer, requires additional parameters
     1. KstemmerDir: Path to directory of data files used by Krovetz's stemmer.
   • "arabic" arabic stemmer, requires additional parameters
     1. arabicStemDir: Path to directory of data files used by the Arabic stemmers.
     2. arabicStemFunc: Which stemming algorithm to apply, one of:
        • arabic_stop : arabic_stop
        • arabic_norm2 : table normalization
        • arabic_norm2_stop : table normalization with stopping
        • arabic_light10 : light9 plus ll prefix
        • arabic_light10_stop : light10 and remove stop words
9. dataFiles: name of file containing list of datafiles to index.

---