This is not a question about software or tools. Please don't comment if you think that's what I'm asking about.

Parsing Unicode-encoded text is a major pain for a software developer, so I thought there must be a better way and quickly came up with an extensible way to do it without too much overhead (no analysis done yet). The scheme depends on no script having more than 127 characters -- without distinction between upper and lower case characters or any special graphical representation of the same basic character (i.e. all of the different ways to write a character based on context, counts as one character).

So, are there any scripts which have more than 127 characters?

  • 8
    I strongly suggest browsing the Unicode book (PDF link). Maybe skim chapters 7~23 first for motivation, then go back to 1~6. I believe you'll find it to be an interesting read. – melboiko Feb 7 '18 at 11:26
  • 34
    Have you, by any chance, reinvented the idea of code pages? – JohannesD Feb 7 '18 at 13:33
  • 18
    I appreciate @bioko suggestion. And I'd give you also this quite interresting read: utf8everywhere.org . Please don't reinvent another "better" standard (relevant xkcd: xkcd.com/927) – Olivier Dulac Feb 7 '18 at 14:58
  • 9
    If you really want to do this (you're not going to get out of it being a "major pain", and your thing will just be a nonstandard major pain) and you're really sure the overhead matters, look into unicode.org/reports/tr6 for something that does essentially the same thing as you're proposing, but uses fixed-size unicode blocks for its 128-character "windows" rather than the use of "scripts" you're proposing. – Random832 Feb 7 '18 at 15:54
  • 9
    Parsing Unicode encoded text is trivial for a software developer who uses an appropriate parsing library. – David Moles Feb 8 '18 at 19:20

Yes, there are.

Most famous is of course the Chinese script with several thousand characters.

For Unicode purposes, Korean also has a lot of characters, because Unicode encodes Korean syllables as one character (and not just the hangul alphabet).

Even the Latin script has surpassed the limit of 127 characters because there are many accented letters (like ä, é, ô, etc.) encoded as single characters, not to mention all those odd additions to the Latin alphabet used for languages where the usual character set wasn't sufficient (e.g. Icelandic Edh and Thorn).

  • More data: According to stackoverflow.com/questions/1366068/…, "Unicode currently has 74605 CJK characters." I admit that this is ~3 scripts, but that's far less than 588 scripts. – Mooing Duck Feb 9 '18 at 2:11
  • Note that you may also argue the Chinese "characters" to be more like "words" or "syllables" of other languages, while the Chinese radicals are more similar to what the westerners call "characters". Though eventually what matters is that it still leads to a similar situation to Korean syllable encoding. – xji Feb 9 '18 at 15:29
  • Native Chinese speaker here :) Note on Japanese and Korean: Japanese has many "loan characters" from (Traditional) Chinese, with some of them modified. Korean doesn't have that many characters, maybe only 80+. – iBug Dec 17 '18 at 14:16

Chinese, Japanese, Sumerian, Akkadian, Egyptian hieroglyphs, Sanskrit, the list is endless.

  • 8
    Sanskrit is a language, not a script. Devanagari (the script) has far fewer than 127 distinct characters; the various ligatures are not encoded separately in Unicode. – chepner Feb 7 '18 at 14:52
  • 3
    @chepner. If we take “charater” to mean akṣara, then you have to count the signs for simple consonants, for consonant clusters, and for either of these followed by a vowel. This will take you well into the hundreds. – fdb Feb 7 '18 at 15:07
  • 5
    That's what I meant by "ligature"; Unicode doesn't encode them, and leaves it as a problem for the rendering engine to handle. – chepner Feb 7 '18 at 15:10
  • 2
    @chepner perhaps too philosophical a question, but isn't any script just a set of ligatures for a long string of ones and zeroes? – John Dvorak Feb 8 '18 at 5:19
  • 5
    @chepner Even if you count Unicode codepoints, there are more than 127 Devanagari characters encoded in Unicode: 128 in the Devanagari block, 30 in the Devanagari Extended block, and 42 in the Vedic Extensions block. – ShreevatsaR Feb 8 '18 at 5:55

For an alphabet used for a single language, Vietnamese has:

  • 29 letters (including the vowels without tone marks)
  • 12 vowels can accept 5 tone marks each
  • All these of course in upper and lower case

For a total of 178 letters.

The Windows-1258 codepage solves this by implementing the tone marks as combining characters only (except for some of the composed caracters that already existed in Latin-1)

For VISCII, the entire upper half is filled with 128 letters, along with six more that replace C0 control characters (a total of 186 letters are encoded, the 178 mentioned, plus the basic ASCII letters not used for Vietnamese).

  • 2
    But the question says: "without distinction between upper and lower case characters". – fdb Feb 7 '18 at 19:15
  • 3
    Oops, I missed that (though I have no idea how his encoding scheme is going to avoid distinguishing upper and lowercase characters) – Random832 Feb 7 '18 at 19:37

Tamil is another example; there are 12 vowels and 18 consonants, plus one special character, so the alphabet has 31 (12+18+1) independent letters and 216 (12 * 16) combined letters (consonant + a vowel mark) for a total of 247 letters. The Wikipedia page link has the Unicode tables for the characters.

  • 2
    But in Unicode, all Indic scripts (including Tamil) are analysed into consonants and vowels (and a virama for vowelessness), saving a lot of characters or code points. – jknappen Feb 7 '18 at 20:44
  • Thanks for the clarification - just wanted the OP to have another interesting alphabet to consider. – Joe McMahon Feb 8 '18 at 19:16

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.