# How can I know if a log-likelihood score is high enough?

I am trying to understand if some terms can be considered as distinctly American or British. So for example I was studying the term "aubergine" and the normalized frequency in the American corpus is of 409 while in British ENglish it is of 3637. I normalized the frequency by making the larger corpus of the same size of the smaller one which is of 34 billion words (since it's my first time using corpora I hope that This is the right process to normalize frequencies: multiply the raw F. by the desired size of the corpus and later divide the end result by the real size of the curpus). Now, what I am trying to understand is if a LL of 2959 is high enough to say that the term aubergine is a key word in British English or not. In the book that I am studying the author uses a cut-off of 50 but since the corpora that I am using are much larger than his I would like to know what cut-off I should use. I am really sorry if I haven't made myself clear but I am very confused because as I said I never used corpora before and also English is not my first language, so I apologize in advance for any mistakes.

For the purpose of normalisation, you should use a formula such as this:

``````normalised_frequency_per_million_words = raw_frequency / corpus_size * 1,000,000
``````

This formula can be adjusted to normalise, for example, per 100,000 words - just replace the 1,000,000 in the formula by 100,000.

When calculating log likelihood for a comparison of the frequency of a linguistic item (e.g. a word) in the two corpora, you should use the raw frequency of the linguistic item and the exact size of the corpora. Do NOT use the normalised figures in the formula. For the interpretation of log likelihood, Paul Rayson gives the following guidelines:

95th percentile; 5% level; p < 0.05; critical value = 3.84

99th percentile; 1% level; p < 0.01; critical value = 6.63

99.9th percentile; 0.1% level; p < 0.001; critical value = 10.83

99.99th percentile; 0.01% level; p < 0.0001; critical value = 15.13

This provides information on the statistical significance of the difference - basically the question of whether with a new corpus you would get a similar result. However, in order to decide if the difference is meaningful, you need to rely on effect size. Is the linguistic item 20% more frequent in corpus A than in corpus B? Or twice as frequent? Whether you consider 20% or even 5% meaningful or not is up to you to decide based on your research question. When comparing two varieties of a language I would not consider a 5% difference to be particularly meaningful.

• I tried to normalize the raw frquency by 1 million but since the corpora that I am using are really big (one of 34 billion words and one of 155 billions) the normalized frequency that I get as a result is an extremely small number and i cannot use them to calculate tha log-likelihood since it is better not to insert numbers that have commas in the calculator that you told me to use (it is the same one that I have been using). This is why I am using that other formula I wrote down in my question. Jan 3, 2019 at 8:53
• Obvioulsy all the words that I am studying are low-frequency (courgette- zucchini, french fries, etc..) so maybe it is not possible to apply the log-likelihood test to them. Since I am writing a thesis about lexical differences between American English and British English I am just trying to understand if it is true that one of the language varieties is beginning to borrow and use some of the food related terms that once belonged to the other language variety, but I'm not sure about how to read the frequency data to state that for ex. the term frenchfries is becoming part of the British lexis Jan 3, 2019 at 8:56
• When calculating log likelihood for a comparison of the frequency of a linguistic item (e.g. a word) in the two corpora, you should use the raw frequency of the linguistic item and the exact size of the corpora. Do NOT use the normalised figures in the formula. The log likelihood test is the appropriate test for your question. Let me know if there is anything else you'd like to know and use the upvote/accept buttons if you're query has been resolved ;) Jan 3, 2019 at 13:11
• I have one more question, so i tried to do what you said with the frequencies for the term "frech fries", the LL resulted is 2691.59, the %diff is 200.97. I don't know how to read these data.. I mean what do I do with the LL, do I only have to read the %diff or do I have to do some other calculation? I'm sorry I feel dumb but I seriously do not understand how this works lol. Thank you so much for your help Jan 4, 2019 at 14:02
• Compare the LL to you got to the numbers mentioned after 'critical value' in the yellow part of the post above. You'll find that your LL is higher than even the highest one mentioned here. Your result is significant with p<0.0001. Jan 4, 2019 at 18:05