I'm a programmer (not a linguist) working on a software product that searches through data using a search term from the user. For English, we require a minimum length of 3 characters for the search term, because there are few useful search terms which are shorter than 3, and those searches tend to return a lot of irrelevant results. I need to identify those information-dense languages where it would be appropriate to reduce the minimum length to 2 characters. (By character, I mean a Unicode UTF-16 character.) At first blush, I'm thinking Chinese, Japanese & Korean. Any advice on how to approach choosing languages with "interesting" 2-character search terms?
-
1I'd suggest the best way to solve this would be empirically with corpora for the different languages, although I appreciate that may not be possible. It also depends to some extent on what is being searched for (personal names, place names, common nouns, etc). Chinese certainly has single-character surnames and some other single-character elements you might wish to search for.– Stuart FCommented Jun 24, 2022 at 14:50
-
1I would guess there are two factors at play: (1) how dense the writing system is and (2) how many different very short words the language allows. A character in Chinese or Japanese is more equivalent to a word, even if the transliterated equivalent might be three-letters or less. I often search for single Chinese characters and don't think any of the software I use limits what can be inputted. Japanese is slightly different because of mixed scripts and different language structure.– VegawatcherCommented Jun 24, 2022 at 15:17
-
2Chinese is the paradigm case of a language with short words, in terms of characters. For English, one would not want to strictly rule out searching for "ox; go; do". It seems to me that considering solutions for Chinese and English would cover any realistically-likely lower limit on characters for a valid search. I suppose it depends on what you mean by "interesting".– user6726Commented Jun 24, 2022 at 15:47
Add a comment
|
1 Answer
Besides the languages already mentioned in the original question I suggest Vietnamese. It has a good wealth of two-Unicode-character words when encoded with precomposed characters (this precautions is important here).
answered Jun 25, 2022 at 20:45
