4

I have zero experience with linguistics. Some friends told me that my question is one in linguistic, so I decided to give a shot here.

Question

While designing a dictionary, people collect words into it. Naively, each word will occupy an entry, but that is not an efficient way. Instead, we should combine words like 'book' and 'books' into an entry, and probably should name that entry 'book' because the word 'book' is more "reduced" than the work 'books'.

In this case, I would like to say that the 'canonical form' of the word 'books' is 'book'. What is the terminology in linguistics (if any) that is similar to my 'canonical form'? And how do people deal with this dictionary-designing problem? Any related terms will also be appreciated. Thank you in advance!

11

The term you are looking for is lemma.
And yes, indeed most linguists will treat book and books as the same "word" (in the sense of word=lemma) in a dictionary, which is why the term "lemma" is sometimes also referred to as the dictionary form of a word. This applies not only when writing a literal dictionary, but is also the common understanding of the term "word" in theoretical linguistics: "books" and "book" are generally regarded as belonging to the same lexicon entry, where "lexicon" here means not a physical lexicon you keep in your bookshelf, but an abstract idea of a language's word inventory that speakers have intuitions about.
I don't know exactly what you'd like to know more about how this problem is dealt with; basically, forms of a word which are mere grammatical variants (such as singular vs. plural in the case of nouns, present vs. past tense in the case of verbs, ...) but carry the same core meaning will be treated as the same "word" (in the sense of lemma), while two words of the same form but with a different meaning (such as the word(s) "bow": to bend down vs. something to shoot an arrow with - both words happen to have the same form, but it makes sense to treat them as different words) are treated as belonging to different lemmas. More difficult cases include phenomena like "parliament" (this could have the sense of building or an institution - is this still in some way the same meaning or should these two senses of "parliament" be treated as different entries in a lexicon?), but this is then more a question of where to draw the line between different "meanings", while "lemma" in the sense of "an abstraction over all grammatical variants of a word" is relatively clear-cut.

BTW: The process of computing the lemma form for a given word (e.g. books -> book), often used in the context of computational linguistics, is called lemmatization.

Another distinction you may find useful in this context is that of types vs. tokens: The sentence "Mary eats a banana and Peter ate a pizza." contains 10 tokens (~= occurrences of strings of characters separated by white space, treating punctuation as separate tokens), but only 9 different types (= distinct word forms abstracted away from different occurrences in the text), since the token "a" occurs twice. Still, "eats" and the past-tense form as "ate" would be treated as different types, although belonging to the same lemma. So including punctuation symbols (which is useful for computational purposes), the sentence contains 10 tokens, 9 types and 8 lemmata.


Re. your questions:

1) The problem with "parliament" is not what is the lemma, but what is the underlying word we want to compute the lemma for at all: if one includes meaning as part of what distinguishes one word from another, where to draw the line between different meanings? But once we decided which words we want to take as words on their own, the question of which is the corresponding lemma is rather clear-cut.
Since a lemma is an abstraction over all grammatical variants of a word and not of a vague concept such as meaning, it is clear which categories are available, and one can simply choose one grammatical value as the default for the lemma form: For nouns, we will usually take nominative singular, for verbs, we will take the infinitive, etc. As soon as we have such a default value, the lemma is clearly decidable. Of course one has to settle these default values themselves; one could try to question why the nominative singular form "book" is more qualified to be the lemma than is the plural form "books", or why we'd choose to use the infinitive to list a verb in a dictionary rather than the present perfect progressive. But speakers have rather consistent intuitions about what is "basic" (for example, imagine if you ask someone "I'm looking for a verb... It's a term for when I investigate or examine something, and I recall it starts with an ''a'", then the other will most likely reply with "You mean 'to analyze'" - and not with "You mean 'analyzed'" or 'analyzes', because they will take the infinitive form as basic: What they're replying with is precisely the lemma), and there are heuristics like complexity (which is the shortest form) or semantic markedness (which is the most unspecialized, generalized form) which help making that decision a lot easier. Since the inflectional paradigm (the set of all grammatical variants of a word) is finite - there is only a finite number of parameters such as case, number etc., and hence only a finite number of combinations of them - the problem of "how do we argue that there's no word more reduced than 'book' in its lemma" can be solved by simply taking a look at all the full paradigm and picking the "most reduced" form among them.

2) If words belong to different parts of speech such as noun ("analysis"), verb ("analyse") and adjective ("analytic"), it is rather universally assumed that they are different words and hence have different lemmata on their own. (Otherwise, if we were to mix words of different categories together, the process of picking the lemma by the pre-set default grammatical value wouldn't work, because there is no nominative singular to "analyse" and no infinitive to "anylsis" etc. - the definition of a lemma entails that it is relative to one part of speech). So the problem of deciding which lemma would be common to all of them doesn't arise in the first place. You will find "analysis", "analyses" and "analytic" listed as individual entries in a dictionary.

-- second update -- 3) I think the closest to what you want are the following two concepts: On the morphological (= form) side, there is the concept of a root (= the indecomposable core part of a word), as you say. On the semantic (= meaning) side, there is the concept of a semanteme (= an indivisible unit of meaning). All three words "analyze", "analysis" and "analytic" will have the same root ("anal" or "analy" or "analyz/analys" - sometimes phonology makes it difficult to settle down to one precise morphological root) and the same semanteme (the abstract meaning that underlines the words in all three categories). While the concept of "lemma" combines form and meaning in its definition, "root" is a purely morphological and "semanteme" a purely semantic concept.
And then there is a whole bunch more concepts like "morpheme" (= an indecomposable, meaning-bearing form unit of a word; this concept subsumes also more functional units like the affixes "-ic" or "un-"), "base" (= the form which affixes attach to, similar to root but doesn't have to be the core part but may already be built up of more than one morpheme), "seme" (= smallest part of a sememe), "sememe" (= a unit of meaning built up from one or more semes, soemewhat the semantic analogon to morpheme) and much more which would exceed the scope of this answer.

If you are further interested, I would suggest you to try an introductory book on morphology or lexical semantics.

9
  • I love this answer! Thanks a lot for the explanation. – Student Jun 7 '19 at 20:11
  • Please allow me to ask more. Specifically, I am curious about the following two questions about lemmata. 1.) How do people know what is the ultimate lemma of a word? Well, this might be too broad to ask, as you have pointed out in the answer that people might not agree with each other (eg for 'parliament'). However, even if I restrict myself in a certain dictionary, it is still not trivial to me. For example, how do we argue that 'book' is more reduced than 'books'; and how do we argue that there's no word more reduced than 'book' in its lemma? – Student Jun 7 '19 at 20:15
  • 2.) Another question is, perhaps there is no ultimate lemma? For example, based on my knowledge, the words 'analysis', 'analyze', 'analytic' should belong to the same lemma. However, while I attempt to pick one of them as a representative of that lemma, I encounter a problem: I don't feel any of them is more reduced than the others! Well, referring to an etymology dictionary or some historic facts might solve this problem, but my impression is that sometimes it is just very unclear. How do people address this problem then? Do dictionary makers nowadays really care about these problems? – Student Jun 7 '19 at 20:18
  • 1
    @Student I edited my post. – lemontree Jun 8 '19 at 12:54
  • 1
    @Studet updated – lemontree Jun 22 '19 at 23:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.