# Collocation: interpreting contingency table for log-likelihood measure

I have a conundrum here.

I'm attempting to recreate the log-likelihood test example in 'The Cambridge Handbook of Corpus Linguistics (2015). However when I run the same test on the same data I get a different result in R.

Chapter 6 on collocation is by Richard Xiao and on page 111 (the end of section 2.2, table 6.1). He does a log-likelihood measure (LL) on data from the BNC. Specifically he uses a LL on the collocation "sweet smell" and on page 109 gives the following figures:

"In the BNC, for example, the frequency counts of sweet and smell are 3,460 and 3,508 respectively in N (98,313,429) tokens, and the two words co-occur 90 times within the 8-word span"

Now back on page 111 he gives the following contingency table:

So if I understand correctly:

a = number of co-occurrences (90)

b = instances of the word 'sweet' (3460)

c = instances of the word 'smell' (3508)

d = total words in the corpus (983131429) - instances of 'sweet' and 'smell' so in total: (98306461)

So then I run the following formula he gives on that same page in R:

``````LL <- 2*(a*log(a)+b*log(b)+c*log(c)+d*log(d)
-(a+b)*log(a+b)-(a+c)*log(a+c)-(b+d)*log(b+d)
-(c+d)*log(c+d)+(a+b+c+d)*log(a+b+c+d))
``````

The LL score I get is ~620, Xiao reports a score of 688. So I wasn't too far off, but why was I off? I checked the syntax of the formula several times, even copied and pasted it out of the e-version of the book and I get the same result. I even tested this against the `LL.collostr`function given in Levshina's 2015 book, which gives me an almost identical value of ~620.

So my only guess is that I'm not interpreting the contingency table correctly. I fiddled around with it a little bit, setting d to the total number of words in the corpus, and a to the combination of all instances of 'sweet' and 'smell' but I get numbers even farther off by doing that, greater than 1000.

So what's going on here? How am I supposed to interpret that contingency table? Xiao doesn't give a lot of details on that table.

Complete code:

``````# III. Log-likelihood

#a = co-occurrence tokens (sweet smell)
a <- 90
#b = word A tokens (sweet)
b <- 3460
#word B tokens (smell)
c <- 3508
#total words in corpus minus A and B
d <- 98313429 - (b+c)

LL1 <- 2*(a*log(a)+b*log(b)+c*log(c)+d*log(d)
-(a+b)*log(a+b)-(a+c)*log(a+c)-(b+d)*log(b+d)
-(c+d)*log(c+d)+(a+b+c+d)*log(a+b+c+d))

library(Rling)

LL2 <- LL.collostr(a, b, c, d)
``````

REFERENCES:

Biber, D., & Reppen, R. (Eds.). (2015). The Cambridge Handbook of English Corpus Linguistics (Cambridge Handbooks in Language and Linguistics). Cambridge: Cambridge University Press. doi:10.1017/CBO9781139764377

Levshina, Natalia. 2015. How to do Linguistics with R: Data exploration and statistical analysis. Amsterdam/Philadelphia: John Benjamins Publishing Co. doi:10.1075/z.195.

• b and c should be 3460 - 90 and 3508 - 90 respectively, since we're looking for instances of sweet without smell and smell without sweet. However, I was unable to replicate either your or Xiao's results either way; I got 1002.147 with your numbers and 1002.135 with mine, whether with your code, with Levshina's, or with a calculator. Switching to 2 or 10 as the base of the logs didn't result in anything close to what you or Xiao got, either. Could you link to your complete R code? Commented Mar 13, 2019 at 9:13
• Question updated with complete code. My first initial tries as well came up with ~1002, I tried the other values, and when I reset them back to what I currently have in the above code, I got the ~620 value. I don't know how that happened. Commented Mar 13, 2019 at 15:20
• You have 60 instead of 90 in your new code. Commented Mar 13, 2019 at 16:10
• No problem :) I'll write a more detailed explanation of contingency tables and the LL measure in an answer later, when I get the time. I looked briefly at the chapter you cite and honestly, I think the exposition was... not so great. Commented Mar 14, 2019 at 9:37
• Could be also a different base of the logarithm: 2 or 10, rather than the natural log used in the OP. Commented Sep 12, 2022 at 9:04

## 1 Answer

There are a couple of typos in the original question, but for future reference:

``````a -> 90
b -> 3_460 - 90 = 3_370
c -> 3_508 - 90 = 3_418

N -> 98_313_429

#N = a + b + c + d
d = N - a - b - c

LL1 <- 2*(
a*log(a)
+b*log(b)
+c*log(c)
+d*log(d)
-(a+b)*log(a+b)
-(a+c)*log(a+c)
-(b+d)*log(b+d)
-(c+d)*log(c+d)
+(N)*log(N))
``````

This gives an output of `1011.422` suggesting either the original references were wrong or the OP has made a typo (very possible given the two typos in the question). The quote is not totally clear as to whether 3,460 means b in total or b not a. However, neither option gives 688.

It's best to refer to the original paper by Dunning who gives the example: `AandB = 110, AnotB = 2442, BnotA = 111, n = 31777 (NotANotB = 29114)` with `LL = 270.72` which indeed the above formula gives.