I want to build a keyword extractor based on the TextRank model as explained in RMPT04. But I don't understand how to calculate the co-occurrence between two words in a window of text explained in the point 3.1. Moreover, is a corpus necessary?

  • 1
    this should be moved to compling or statistics.
    – jlawler
    Commented May 5, 2013 at 16:07
  • 1
    @jlawler: I disagree. There is no compling. nlp is a rather neglected topic on statistics.SE. Until NLP.SE starts, this question should be considered on-topic here.
    – prash
    Commented May 6, 2013 at 14:49
  • @ArnoldPaye: The "window" here is just an indication of how many intermediate words (in a sentence) are to be ignored in order to consider two words as "co-ocurring". Given the sentence, "Mary had a little lamb", "Mary" and "lamb" are considered to co-occur of the window is >=4. They are considered as not co-occurring if the window is <4. These co-occurrence counts are extracted over a corpus: a corpus is necessary.
    – prash
    Commented May 6, 2013 at 15:37
  • @prash Whether NLP SE starts or not, it doesn't influence our site. NLP questions will always be on topic on this site regardless of that site existence. If we had to cut topics that overlapped, most sites would had a really narrow field of expertise in their FAQ. :)
    – Alenanno
    Commented Jun 26, 2013 at 13:49
  • Note that the link to the referenced article may go stale making this question void. Please add enough context to make the question self-contained. Commented Mar 13, 2017 at 14:05

1 Answer 1


Of course you need a corpus.

Generally in statistical NLP, you train your model based on a corpus. For example, for text classification where an input document is fed to the model and it should output its class (from a list of classes). The model is trained on many documents with their corresponding classes and when the new document is tested under that model, it will use the features (information) which was extracted from those documents to classify the new document.

In the case of co-occurrence of two words, you can use context-vector, which is very common in statistical NLP. It has a simple definition and very easy to implement, but you will need a corpus:

You will define a vector with fixed length (the number of unique words in your corpus) for each unique word in your corpus. The context vector for each word tells us how many times other words have co-occurred with the current word in the defined window, e.g. in a window of words, you see what are the other words occurred with the current word and increment their corresponding element in the context vector. A simple example is show below:

Corpus: A D C E A D F E B A C E D

Window size: 2 (the 2 words of the either side)

Context vectors:

      A B C D E

A     0 1 3 2 3    
B     1 0 1 0 1     
C     3 1 0 2 2    
D     2 0 2 0 4    
E     3 1 2 4 0

Using these context vectors you can get co-occurrences very easy. For example co-occurrence of D and E is D[E] = 4.

  • sorry but I dont get how D[E] got 4 and also others like A[E] gets 3?
    – samsamara
    Commented Apr 28, 2015 at 2:44
  • @KillBill If you consider the window around all the occurrences of A, you will find 3 E's co-occuring with A. The same thing goes with D, if you consider the window around each occurrence of D, and count all the occurrences of E in that windows, you'll find 4 E's. Did you get it?
    – Moh
    Commented Apr 28, 2015 at 8:26
  • Given the window size is 2 and if i consider all the occurrences of A, I see only "E A" in the corpus that occurs with A? at 4 and 5 indeces
    – samsamara
    Commented Apr 28, 2015 at 8:54
  • @KillBill: I've said the window size is 2 of either size, I mean the window is [-2,+2]. Is it clear now?
    – Moh
    Commented Apr 28, 2015 at 9:07
  • 1
    very helpful explanation Commented Nov 16, 2017 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.