11088368 bytes used for LogProb tables (C++ version)
Building integer logs conversion tables
The following options are from the config file and will be overwritten by any command line options.
Parameter 'c' changed from '' to '/SMT/data/TrainSample.en_TrainSample.ch.snt'
Parameter 'l' changed from '107-11-15.152422.qing.log' to '107-10-29.154814.nbach.log'
Parameter 'm3' changed from '5' to '3'
Parameter 'm4' changed from '5' to '3'
Parameter 'model1dumpfrequency' changed from '0' to '1'
Parameter 'model4smoothfactor' changed from '0.2' to '0.4'
Parameter 'nsmooth' changed from '64' to '4'
Parameter 'o' changed from '107-11-15.152422.qing' to '/SMT/data/giza.ch-en/'
Parameter 'p0' changed from '-1' to '0.999'
Parameter 's' changed from '' to '/SMT/data/TrainSample.en.vcb'
Parameter 't' changed from '' to './TrainSample.ch.vcb'
Parameter 'targetvocabularyfile' changed from './TrainSample.ch.vcb' to '/SMT/data/TrainSample.ch.vcb'
Parameter 'outputdiff' changed from '' to '/tmp/output001'
general parameters:
-------------------
ml = 101  (maximum sentence length)

No. of iterations:
-------------------
currmodel1iterations = 1  (current run of iterations for Model 1)
hmmiterations = 5  (mh)
model1iterations = 5  (number of iterations for Model 1)
model2iterations = 0  (number of iterations for Model 2)
model3iterations = 3  (number of iterations for Model 3)
model4iterations = 3  (number of iterations for Model 4)
model5iterations = 0  (number of iterations for Model 5)
model6iterations = 0  (number of iterations for Model 6)

parameter for various heuristics in GIZA++ for efficient training:
------------------------------------------------------------------
countincreasecutoff = 1e-06  (Counts increment cutoff threshold)
countincreasecutoffal = 1e-05  (Counts increment cutoff threshold for alignments in training of fertility models)
mincountincrease = 1e-07  (minimal count increase)
peggedcutoff = 0.03  (relative cutoff probability for alignment-centers in pegging)
probcutoff = 1e-07  (Probability cutoff threshold for lexicon probabilities)
probsmooth = 1e-07  (probability smoothing (floor) value )

parameters for describing the type and amount of output:
-----------------------------------------------------------
compactalignmentformat = 0  (0: detailled alignment format, 1: compact alignment format )
hmmdumpfrequency = 0  (dump frequency of HMM)
l = 107-10-29.154814.nbach.log  (log file name)
log = 0  (0: no logfile; 1: logfile)
model1dumpfrequency = 1  (dump frequency of Model 1)
model2dumpfrequency = 0  (dump frequency of Model 2)
model345dumpfrequency = 0  (dump frequency of Model 3/4/5)
nbestalignments = 0  (for printing the n best alignments)
nodumps = 0  (1: do not write any files)
o = /SMT/data/giza.ch-en/  (output file prefix)
onlyaldumps = 0  (1: do not write any files)
outputdiff = /tmp/output001  (Diffence file to output)
outputpath =   (output path)
transferdumpfrequency = 0  (output: dump of transfer from Model 2 to 3)
verbose = 0  (0: not verbose; 1: verbose)
verbosesentence = -10  (number of sentence for which a lot of information should be printed (negative: no output))

parameters describing input files:
----------------------------------
c = /SMT/data/TrainSample.en_TrainSample.ch.snt  (training corpus file name)
d =   (dictionary file name)
prevtmodel =   (Previous_TTable)
s = /SMT/data/TrainSample.en.vcb  (source vocabulary file name)
t = /SMT/data/TrainSample.ch.vcb  (target vocabulary file name)
tc =   (test corpus file name)

smoothing parameters:
---------------------
emalsmooth = 0.2  (f-b-trn: smoothing factor for HMM alignment model (can be ignored by -emSmoothHMM))
model23smoothfactor = 0  (smoothing parameter for IBM-2/3 (interpolation with constant))
model4smoothfactor = 0.4  (smooting parameter for alignment probabilities in Model 4)
model5smoothfactor = 0.1  (smooting parameter for distortion probabilities in Model 5 (linear interpolation with constant))
nsmooth = 4  (smoothing for fertility parameters (good value: 64): weight for wordlength-dependent fertility parameters)
nsmoothgeneral = 0  (smoothing for fertility parameters (default: 0): weight for word-independent fertility parameters)

parameters modifying the models:
--------------------------------
compactadtable = 1  (1: only 3-dimensional alignment table for IBM-2 and IBM-3)
deficientdistortionforemptyword = 0  (0: IBM-3/IBM-4 as described in (Brown et al. 1993); 1: distortion model of empty word is deficient; 2: distoriton model of empty word is deficient (differently); setting this parameter also helps to avoid that during IBM-3 and IBM-4 training too many words are aligned with the empty word)
depm4 = 76  (d_{=1}: &1:l, &2:m, &4:F, &8:E, d_{>1}&16:l, &32:m, &64:F, &128:E)
depm5 = 68  (d_{=1}: &1:l, &2:m, &4:F, &8:E, d_{>1}&16:l, &32:m, &64:F, &128:E)
emalignmentdependencies = 2  (lextrain: dependencies in the HMM alignment model.  &1: sentence length; &2: previous class; &4: previous position;  &8: French position; &16: French class)
emprobforempty = 0.4  (f-b-trn: probability for empty word)

parameters modifying the EM-algorithm:
--------------------------------------
m5p0 = -1  (fixed value for parameter p_0 in IBM-5 (if negative then it is determined in training))
manlexfactor1 = 0  ()
manlexfactor2 = 0  ()
manlexmaxmultiplicity = 20  ()
maxfertility = 10  (maximal fertility for fertility models)
p0 = 0.999  (fixed value for parameter p_0 in IBM-3/4 (if negative then it is determined in training))
pegging = 0  (0: no pegging; 1: do pegging)

general parameters:
-------------------
ml = 101  (maximum sentence length)

No. of iterations:
-------------------
currmodel1iterations = 1  (current run of iterations for Model 1)
hmmiterations = 5  (mh)
model1iterations = 5  (number of iterations for Model 1)
model2iterations = 0  (number of iterations for Model 2)
model3iterations = 3  (number of iterations for Model 3)
model4iterations = 3  (number of iterations for Model 4)
model5iterations = 0  (number of iterations for Model 5)
model6iterations = 0  (number of iterations for Model 6)

parameter for various heuristics in GIZA++ for efficient training:
------------------------------------------------------------------
countincreasecutoff = 1e-06  (Counts increment cutoff threshold)
countincreasecutoffal = 1e-05  (Counts increment cutoff threshold for alignments in training of fertility models)
mincountincrease = 1e-07  (minimal count increase)
peggedcutoff = 0.03  (relative cutoff probability for alignment-centers in pegging)
probcutoff = 1e-07  (Probability cutoff threshold for lexicon probabilities)
probsmooth = 1e-07  (probability smoothing (floor) value )

parameters for describing the type and amount of output:
-----------------------------------------------------------
compactalignmentformat = 0  (0: detailled alignment format, 1: compact alignment format )
hmmdumpfrequency = 0  (dump frequency of HMM)
l = 107-10-29.154814.nbach.log  (log file name)
log = 0  (0: no logfile; 1: logfile)
model1dumpfrequency = 1  (dump frequency of Model 1)
model2dumpfrequency = 0  (dump frequency of Model 2)
model345dumpfrequency = 0  (dump frequency of Model 3/4/5)
nbestalignments = 0  (for printing the n best alignments)
nodumps = 0  (1: do not write any files)
o = /SMT/data/giza.ch-en/  (output file prefix)
onlyaldumps = 0  (1: do not write any files)
outputdiff = /tmp/output001  (Diffence file to output)
outputpath =   (output path)
transferdumpfrequency = 0  (output: dump of transfer from Model 2 to 3)
verbose = 0  (0: not verbose; 1: verbose)
verbosesentence = -10  (number of sentence for which a lot of information should be printed (negative: no output))

parameters describing input files:
----------------------------------
c = /SMT/data/TrainSample.en_TrainSample.ch.snt  (training corpus file name)
d =   (dictionary file name)
prevtmodel =   (Previous_TTable)
s = /SMT/data/TrainSample.en.vcb  (source vocabulary file name)
t = /SMT/data/TrainSample.ch.vcb  (target vocabulary file name)
tc =   (test corpus file name)

smoothing parameters:
---------------------
emalsmooth = 0.2  (f-b-trn: smoothing factor for HMM alignment model (can be ignored by -emSmoothHMM))
model23smoothfactor = 0  (smoothing parameter for IBM-2/3 (interpolation with constant))
model4smoothfactor = 0.4  (smooting parameter for alignment probabilities in Model 4)
model5smoothfactor = 0.1  (smooting parameter for distortion probabilities in Model 5 (linear interpolation with constant))
nsmooth = 4  (smoothing for fertility parameters (good value: 64): weight for wordlength-dependent fertility parameters)
nsmoothgeneral = 0  (smoothing for fertility parameters (default: 0): weight for word-independent fertility parameters)

parameters modifying the models:
--------------------------------
compactadtable = 1  (1: only 3-dimensional alignment table for IBM-2 and IBM-3)
deficientdistortionforemptyword = 0  (0: IBM-3/IBM-4 as described in (Brown et al. 1993); 1: distortion model of empty word is deficient; 2: distoriton model of empty word is deficient (differently); setting this parameter also helps to avoid that during IBM-3 and IBM-4 training too many words are aligned with the empty word)
depm4 = 76  (d_{=1}: &1:l, &2:m, &4:F, &8:E, d_{>1}&16:l, &32:m, &64:F, &128:E)
depm5 = 68  (d_{=1}: &1:l, &2:m, &4:F, &8:E, d_{>1}&16:l, &32:m, &64:F, &128:E)
emalignmentdependencies = 2  (lextrain: dependencies in the HMM alignment model.  &1: sentence length; &2: previous class; &4: previous position;  &8: French position; &16: French class)
emprobforempty = 0.4  (f-b-trn: probability for empty word)

parameters modifying the EM-algorithm:
--------------------------------------
m5p0 = -1  (fixed value for parameter p_0 in IBM-5 (if negative then it is determined in training))
manlexfactor1 = 0  ()
manlexfactor2 = 0  ()
manlexmaxmultiplicity = 20  ()
maxfertility = 10  (maximal fertility for fertility models)
p0 = 0.999  (fixed value for parameter p_0 in IBM-3/4 (if negative then it is determined in training))
pegging = 0  (0: no pegging; 1: do pegging)
Reading vocabulary file from:/SMT/data/TrainSample.en.vcb
Reading vocabulary file from:/SMT/data/TrainSample.ch.vcb

reading vocabulary files 
Source vocabulary list has 9674 unique tokens 
Target vocabulary list has 11912 unique tokens 
Calculating vocabulary frequencies from corpus /SMT/data/TrainSample.en_TrainSample.ch.snt
Reading more sentence pairs into memory ... 
Corpus fits in memory, corpus has: 5000 sentence pairs.
 Train total # sentence pairs (weighted): 5000
Size of source portion of the training corpus: 174531 tokens
Size of the target portion of the training corpus: 136768 tokens 
In source portion of the training corpus, only 9673 unique tokens appeared
In target portion of the training corpus, only 11910 unique tokens appeared
lambda for PP calculation in IBM-1,IBM-2,HMM:= 136768/(179531-5000)== 0.783632
==========================================================
Model1 Training Started at: Thu Nov 15 15:24:23 2007

-----------
Model1: Iteration 1
Partial Model1 Training took: 8 seconds

Entire Training took: 19 seconds
Program Finished at: Thu Nov 15 15:24:41 2007

==========================================================
