2

I have a conundrum here.

I'm attempting to recreate the log-likelihood test example in 'The Cambridge Handbook of Corpus Linguistics (2015). However when I run the same test on the same data I get a different result in R.

Chapter 6 on collocation is by Richard Xiao and on page 111 (the end of section 2.2, table 6.1). He does a log-likelihood measure (LL) on data from the BNC. Specifically he uses a LL on the collocation "sweet smell" and on page 109 gives the following figures:

"In the BNC, for example, the frequency counts of sweet and smell are 3,460 and 3,508 respectively in N (98,313,429) tokens, and the two words co-occur 90 times within the 8-word span"

Now back on page 111 he gives the following contingency table:

enter image description here

So if I understand correctly:

a = number of co-occurrences (90)

b = instances of the word 'sweet' (3460)

c = instances of the word 'smell' (3508)

d = total words in the corpus (983131429) - instances of 'sweet' and 'smell' so in total: (98306461)

So then I run the following formula he gives on that same page in R:

LL <- 2*(a*log(a)+b*log(b)+c*log(c)+d*log(d)
         -(a+b)*log(a+b)-(a+c)*log(a+c)-(b+d)*log(b+d)
         -(c+d)*log(c+d)+(a+b+c+d)*log(a+b+c+d))

The LL score I get is ~620, Xiao reports a score of 688. So I wasn't too far off, but why was I off? I checked the syntax of the formula several times, even copied and pasted it out of the e-version of the book and I get the same result. I even tested this against the LL.collostrfunction given in Levshina's 2015 book, which gives me an almost identical value of ~620.

So my only guess is that I'm not interpreting the contingency table correctly. I fiddled around with it a little bit, setting d to the total number of words in the corpus, and a to the combination of all instances of 'sweet' and 'smell' but I get numbers even farther off by doing that, greater than 1000.

So what's going on here? How am I supposed to interpret that contingency table? Xiao doesn't give a lot of details on that table.

Complete code:

# III. Log-likelihood

#a = co-occurrence tokens (sweet smell)
a <- 90
#b = word A tokens (sweet)
b <- 3460
#word B tokens (smell)
c <- 3508
#total words in corpus minus A and B
d <- 98313429 - (b+c)

LL1 <- 2*(a*log(a)+b*log(b)+c*log(c)+d*log(d)
         -(a+b)*log(a+b)-(a+c)*log(a+c)-(b+d)*log(b+d)
         -(c+d)*log(c+d)+(a+b+c+d)*log(a+b+c+d))

library(Rling)

LL2 <- LL.collostr(a, b, c, d)

REFERENCES:

Biber, D., & Reppen, R. (Eds.). (2015). The Cambridge Handbook of English Corpus Linguistics (Cambridge Handbooks in Language and Linguistics). Cambridge: Cambridge University Press. doi:10.1017/CBO9781139764377

Levshina, Natalia. 2015. How to do Linguistics with R: Data exploration and statistical analysis. Amsterdam/Philadelphia: John Benjamins Publishing Co. doi:10.1075/z.195.

6
  • 1
    b and c should be 3460 - 90 and 3508 - 90 respectively, since we're looking for instances of sweet without smell and smell without sweet. However, I was unable to replicate either your or Xiao's results either way; I got 1002.147 with your numbers and 1002.135 with mine, whether with your code, with Levshina's, or with a calculator. Switching to 2 or 10 as the base of the logs didn't result in anything close to what you or Xiao got, either. Could you link to your complete R code? Mar 13, 2019 at 9:13
  • Question updated with complete code. My first initial tries as well came up with ~1002, I tried the other values, and when I reset them back to what I currently have in the above code, I got the ~620 value. I don't know how that happened.
    – Wangana
    Mar 13, 2019 at 15:20
  • 1
    You have 60 instead of 90 in your new code. Mar 13, 2019 at 16:10
  • 1
    No problem :) I'll write a more detailed explanation of contingency tables and the LL measure in an answer later, when I get the time. I looked briefly at the chapter you cite and honestly, I think the exposition was... not so great. Mar 14, 2019 at 9:37
  • 1
    Could be also a different base of the logarithm: 2 or 10, rather than the natural log used in the OP.
    – Roger V.
    Sep 12, 2022 at 9:04

1 Answer 1

1

There are a couple of typos in the original question, but for future reference:

a -> 90  
b -> 3_460 - 90 = 3_370
c -> 3_508 - 90 = 3_418

N -> 98_313_429 

#N = a + b + c + d
d = N - a - b - c 

LL1 <- 2*(
          a*log(a)
         +b*log(b)
         +c*log(c)
         +d*log(d)
         -(a+b)*log(a+b)
         -(a+c)*log(a+c)
         -(b+d)*log(b+d)
         -(c+d)*log(c+d)
         +(N)*log(N))

This gives an output of 1011.422 suggesting either the original references were wrong or the OP has made a typo (very possible given the two typos in the question). The quote is not totally clear as to whether 3,460 means b in total or b not a. However, neither option gives 688.

It's best to refer to the original paper by Dunning who gives the example: AandB = 110, AnotB = 2442, BnotA = 111, n = 31777 (NotANotB = 29114) with LL = 270.72 which indeed the above formula gives.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.