Hi my country's language is not in nltk in python. I have a wordlist that contains word and part of speech (noun, verd adj etc.) in excel. But I don't know how to build a corpus. My language is not in python nltk. How to build tag pos for my language in nltk in python?

  • This is rather broad and vague. There is a number of questions about this over on Stack Overflow. Having tags for functional words (prepositions, particles, adverbs etc) would be very important; the open word classes probably cannot ever be fully populated anyway. – tripleee Jun 13 '18 at 6:44
  • 2
    A common way is to create a text file with tagged text, and training the NLTK tagger on that. See e.g. stackoverflow.com/a/44475198/874188 ... and probably avoid Excel for anything related to words and text. – tripleee Jun 13 '18 at 6:45
  • Thank u. So I can't use nltk ect other tools. Because my language is not in nltk, other tools.language is sov. Mongolia language. So I can't tag pos sentence. So What should I do first. How to do? Please recommend me – Baatarhuu Monhkbayar Jun 14 '18 at 11:41
  • I'm afraid I'm not able to understand your comment. Why can't you use NLTK? Why can't you create a tagged corpus and train the NLTK tagger on that? – tripleee Jun 14 '18 at 11:55
  • Thank u replied me. Because I don't understand tagged corpus. How to build?. I could use nltk to any other than English? – Baatarhuu Monhkbayar Jun 14 '18 at 12:12

Chapter 5 of the online NLTK book explains the concepts and procedures you would use to create a tagged corpus.

There are several taggers which can use a tagged corpus to build a tagger for a new language. You will probably want to experiment with at least a few of them.

A tagged corpus is better than just a list of words because many languages have ambiguities, and working with a large enough collection of representative samples allows you to cope with this.

For example, an English lexicon would simply tell you that "to" has two possible POS tags (infinitive marker or preposition) but given enough examples in context, training a tagger hopefully sorts out which reading is correct in which context. The training might establish that "to" before an uninflected verb gets the INF tag, and assign PREP elsewhere (though reality is somewhat more complex than that).

If your language has a radically different orthography than the NLTK taggers are constructed to cope with, this is probably too simplistic, but assuming you are dealing with something resembling Latin or Cyrillic script, the examples in the book should be straightforward to apply to your language.

See also chapter 11 which focuses more on the collection and management side of the corpus process.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.