Question about ngram models, might be a stupid question:
With ngram models, the probability of a sequence is the product of the conditional probabilities of the n-grams into which the sequence can be decomposed (I'm going by the n-gram chapter in Jurafsky and Martin's book Speech and Language Processing here). So if we were to calculate the probability of 'I like cheese' using bigrams:
Pr(I like cheese) = Pr(like | I) x Pr(cheese | like)
So if the probability that 'like' appears after 'I' is very high, and the probability 'cheese' appears after 'like' is very high, then the sequence 'I like cheese' will also have a very high probability. Suppose 'I' appears just 3 times in the corpus, 'I like' appears 2 times, 'like' appears 4 times and 'like cheese' appears 3 times, then Pr(like | I) = 0.67, Pr(cheese | like) = 0.75, and Pr(I like cheese) = 0.5025.
What does it mean to say Pr(I like cheese) = 0.5025? Clearly it cannot mean that around half the sequences in the corpus will be 'I like cheese', since the bigrams which compose 'I like cheese' do not need to appear loads and loads for them to have a high conditional probability. Does Pr(I like cheese) = 0.5025 just mean 'I like cheese' is likely to appear in the corpus, even if it just appears once?