I just used OpenNLP for a small program where I was supposed to segment a paragraph into sentences.

Although I was able to complete the task after reading some documentation and going through their test cases, I couldn't help but notice that I still have to train for all the abbreviations (example Yahoo!), even when I created a custom abbreviation dictionary, passed it to SentenceDetectorFactory and used it to train SentenceDetectorME.

I am using similar approach as used in this test case.

I couldn't find this behaviour in their documentation nor I could find any explanation.

Is there something I am missing or is it required to train the model on abbreviations as well? If it is required to train the model on abbreviations then why specify them separately?

Edit: I am using version 1.5.3-rc3.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.