I would like to find the likelihood of a sequence of characters (the test data), given two unigram models.

The sequence (test data) is:


The models are:

       Model 1       Model 2
P(A)     0.3           0.4  
P(B)     0.4           0.5  
P(C)     0.3           0.1

Basically, I would like to know the likelihood, and if I can make a prediction as to which model the sequence belong and the underlying assumptions. I understand that given any unigram language model, the likelihood (or probability) of any sequence of characters is p(sequence of characters|Model).

What I have done so far was to find the MLE for each character:

P(A) =1/5 ; P(B) = 2/5 ; P(C) = 1/5

I don't know how to compute p(sequence of characters|Model). Should I multiply these to find the likelihood and establish which model it came from? How to handle the model probabilities given?

Thanks in advance :-)


Unigram models tend to rely on the naïve Bayes assumption: that is, they assume that every character is independent of everything else around it.

Under this assumption, the likelihood of A B C B B is just the product of the individual probabilities for each letter.

This assumption, as it turns out, isn't a very good one—which is why we move up to more powerful n-gram models.

  • Thanks. But given the likelihood is p(sequence of characters|Model), how does this product of the individual probabilities determine which model would predict it? – user102859 Jan 27 '19 at 19:29
  • @feijao P(seq|model) = P(first|model) P(second|model) P(third|model) etc etc. From this, and Bayes' Rule, you can get P(model|seq). Or you can just pick whichever has the highest likelihood, which is called Maximum Likelihood Estimation. – Draconis Jan 27 '19 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.