I am exploring the financial-related dictionary of Loughran and McDonald. I want to calculate the TF-IDF weighting of the words within the dictionary. I use the following formula, based on the paper "When Is a Liability Not a Liability":

enter image description here

Q1: Which value should I use for N (the total number of 10-Ks in the sample)? If I use the cleaned (50,115) or the raw (121,217) number of 10-Ks, I get negative values for some words. For example, the word “almost” has a value of 126,738 for Doc Count. However, log(121,217 / 126,738) yields a negative value which is not possible.

Q2: What does column E and F of the master dictionary represent? Are these values equal to tfi,j the raw count of the ith word in the j-th document, and aj the average word count in the document (page 42 in the paper)?

Thank you


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.