3

A lexeme to me has always been a fairly abstract entity and a lemma a concrete form that is often used to represent an lexeme.

It is surprising to me, then, that the process in NLP is almost always called lemmatization(400k ghits) rather than lexemization(840 ghits) or lexematization(128 ghits) (a search at my university’s library reveals a similiar difference of 2445:1:1) While I can understand that if an analysis arrives at a lemma by way of various rules similar to destemming applied to a text string it may be the most appropriate term, at a certain point (for instance, once an abstract entity in a database, or attached with some other metadata such as part of speech), shouldn't the term be instead lexeme?

Is there a particular reason that the term lemma seems to be virtually universally used in much of NLP literature to mean both lemma and lexeme?

Improve this question

1 Answer 1

Reset to default
2

Well, there are differences between lemma and lexeme in NLP. Lemmatisation may tell you that some lemma is bank but you need another process (word sense disambiguation) to discriminate between bank (of a river) and bank (where you put money). There are also multi word expressions (MWEs) that count as multiple lemmas but may count as a single lexeme.

Improve this answer
1
  • I guess that makes sense. In my mind, it seems fairly easy enough to determine the POS (at least for the language I'm working on), hence distinguishing bank (a noun) from bank (a verb). That seemed to be starting to cross the threshold between the two, but indeed I didn't consider the multiple lexemes for a single POS which would require a second pass
    – user0721090601
    Feb 5, 2020 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.