I am working through an exercise where, given a set of corpora, I will implement a simple model on a test corpus to determine the most likely corpus.

Say the corpora with which I want to learn are the following (could be anything but I'm just using these":

  • this corpus is all lowercase
  • This Corpus Capitalized The First Letters
  • thiS corpuS cApitaLIzes leTTErs at RANDOM

Now, I would break apart each corpus into bigram pairs, e.g (first):

  • {" T", "HI",...,"PS","S "}

I would find the counts of:

  • each letter bigram
  • each letter unigram

I would then gather the probabilities, where "HI" would be:

 P(I | H) = Count(HI)/Count(H)

I would then take a testing sentence and break it apart into bigram as above. I would then check the probability of each bigram vs the probabilities of the corpora, adding each up and normalizing. The one with the highest probability would be a predicted corpus.

I realize this looks very simple. It seems too simple to me. I just want to make sure it would do what I think it would. Does anybody have any criticisms or critiques of my methodology?

I eventually would like to try this out in python as a means of learning that language as well.

  • Your method is unclear. What is the order of your quantifiers? Why do you care about the conditional P(H | I) instead of just P("HI")? What are you adding up and then what normalizing against? It seem like you have the elements but are putting them together in unexpected ways. You should discuss this with your class's TA.
    – Mitch
    Oct 4, 2016 at 16:52
  • Is there a difference between the two? I am adding up each bigram probability according to above and normalizing against the number of letter bigrams per test sentence. The results look fairly good, maybe false positives?
    – Héctor
    Oct 4, 2016 at 22:45
  • Is there a difference between the two what? If the results look good then I suppose what ever is going on must be OK. Try on data outside of what's given to you to check.
    – Mitch
    Oct 5, 2016 at 1:05


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.