I have a conundrum here.

I'm attempting to recreate the log-likelihood test example in 'The Cambridge Handbook of Corpus Linguistics (2015). However when I run the same test on the same data I get a different result in R.

Chapter 6 on collocation is by Richard Xiao and on page 111 (the end of section 2.2, table 6.1). He does a log-likelihood measure (LL) on data from the BNC. Specifically he uses a LL on the collocation "sweet smell" and on page 109 gives the following figures:

"In the BNC, for example, the frequency counts of sweet and smell are 3,460 and 3,508 respectively in N (98,313,429) tokens, and the two words co-occur 90 times within the 8-word span"

Now back on page 111 he gives the following contingency table:

enter image description here

So if I understand correctly:

a = number of co-occurrences (90)

b = instances of the word 'sweet' (3460)

c = instances of the word 'smell' (3508)

d = total words in the corpus (983131429) - instances of 'sweet' and 'smell' so in total: (98306461)

So then I run the following formula he gives on that same page in R:

LL <- 2*(a*log(a)+b*log(b)+c*log(c)+d*log(d)

The LL score I get is ~620, Xiao reports a score of 688. So I wasn't too far off, but why was I off? I checked the syntax of the formula several times, even copied and pasted it out of the e-version of the book and I get the same result. I even tested this against the LL.collostrfunction given in Levshina's 2015 book, which gives me an almost identical value of ~620.

So my only guess is that I'm not interpreting the contingency table correctly. I fiddled around with it a little bit, setting d to the total number of words in the corpus, and a to the combination of all instances of 'sweet' and 'smell' but I get numbers even farther off by doing that, greater than 1000.

So what's going on here? How am I supposed to interpret that contingency table? Xiao doesn't give a lot of details on that table.

Complete code:

# III. Log-likelihood

#a = co-occurrence tokens (sweet smell)
a <- 90
#b = word A tokens (sweet)
b <- 3460
#word B tokens (smell)
c <- 3508
#total words in corpus minus A and B
d <- 98313429 - (b+c)

LL1 <- 2*(a*log(a)+b*log(b)+c*log(c)+d*log(d)


LL2 <- LL.collostr(a, b, c, d)


Biber, D., & Reppen, R. (Eds.). (2015). The Cambridge Handbook of English Corpus Linguistics (Cambridge Handbooks in Language and Linguistics). Cambridge: Cambridge University Press. doi:10.1017/CBO9781139764377

Levshina, Natalia. 2015. How to do Linguistics with R: Data exploration and statistical analysis. Amsterdam/Philadelphia: John Benjamins Publishing Co. doi:10.1075/z.195.

  • 1
    b and c should be 3460 - 90 and 3508 - 90 respectively, since we're looking for instances of sweet without smell and smell without sweet. However, I was unable to replicate either your or Xiao's results either way; I got 1002.147 with your numbers and 1002.135 with mine, whether with your code, with Levshina's, or with a calculator. Switching to 2 or 10 as the base of the logs didn't result in anything close to what you or Xiao got, either. Could you link to your complete R code? – WavesWashSands Mar 13 '19 at 9:13
  • Question updated with complete code. My first initial tries as well came up with ~1002, I tried the other values, and when I reset them back to what I currently have in the above code, I got the ~620 value. I don't know how that happened. – Wangana Mar 13 '19 at 15:20
  • 1
    You have 60 instead of 90 in your new code. – WavesWashSands Mar 13 '19 at 16:10
  • ah, so I did, that explains where I got the 620 from, thank you for pointing that out. – Wangana Mar 13 '19 at 18:45
  • 1
    No problem :) I'll write a more detailed explanation of contingency tables and the LL measure in an answer later, when I get the time. I looked briefly at the chapter you cite and honestly, I think the exposition was... not so great. – WavesWashSands Mar 14 '19 at 9:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.