3

I would like to normalize the result of Jiang & Conrath similarity measure by dividing it by the maximum value(normally should be when both synsets are the same). I am using the implementation of Jiang & Conrath similarity in wordnet. when I use two same synsets for example slap.jcn_similarity(slap, brown_ic) the answer is 1e+300. How should I deal with it to normalize the results? what is the range of Jiang & Conrath similarity?

I read in this website(http://maraca.d.umn.edu/similarity/measures.html):

There are two special cases that need to be handled carefully when computing relatedness; both of these involve the case when jcn_distance is zero.

In the first case, we have ic(synset1) = ic(synset2) = ic(lcs) = 0. In an ideal world, this would only happen when all three concepts, viz. synset1, synset2, and lcs, are the root node. However, when a synset has a frequency count of zero, we use the value 0 for the information content. In this first case, we return 0 due to lack of data.

In the second case, we have ic(synset1) + ic(synset2) = 2 * ic(ics). This is almost always found when synset1 = synset2 = lcs (i.e., the two input synsets are the same). Intuitively this is the case of maximum relatedness, which would be infinity, but it is impossible to return infinity. Insteady we find the smallest possible distance greater than zero and return the multiplicative inverse of that distance.

I actually could not figure out the first case exactly. Is this happen when a word is not available in the corpus for example the word "wwsds"? How should we deal with these two cases in python programming? I would be grateful if anyone can explain any python program for dealing with these cases.

1 Answer 1

1

You can rewrite Jiang & Conrath similarity by: 1 / log(jcn_distance). If jcn_distance is equal to 0, then by adding 'log' the result will be 0.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.