I want to do some smoothing on a corpus and compare the results. for train language model without smoothing I used "-addsmooth 0" option now I should add two different smoothing and compare them, I couldn't find any command for "Good Turing" and "Kneser-Ney" Smoothing . I found somewhere this command :
ngram-count - vocab Lexicon2003-72k.txt
-read CNA0001-2M.count
-order 3
-lm CNA0001-2M_N3_KN.lm
-kndiscount1
-kndiscount2
-kndiscount3
but it is modified kneser-ney and I cant understand what is different between -kndiscount1
and -kndiscount2
for example. what is the meaning of that number?(1,2)
and why we should used them together?
I don't want to use interpolation.