4

I'm currently in Thailand and a bit frustrated with the difficult writing system here. Normally after a week in a country I can learn the writing system to a functional degree (except Han characters of course).

I can draw the characters just fine but many characters look alike and many characters represent the same sound, but the two sets of "alikeness" are totally independent and unrelated!

Therefore I'd like to learn the most frequent letters first. But short of wandering around in the heat and humidity with a notepad and building a frequency table myself, I'm sure there must be a letter frequency list out there for Thai already. Can somebody point me to it?

Improve this question
3
  • In general these are they types of statistics you need to compile yourself. Heck, I'm a native English speaker but the only reason I know the most common letters in English is due to Wheel of Fortune. If you have some basic programming skills it shouldn't be too hard to copy and paste a few Thai newspaper articles into a file and then write a script to compute the frequencies.
    – acattle
    Commented Aug 30, 2013 at 16:20
  • Yeah I've done it before and if I wasn't backpacking around right now I probably would just do it myself. Still plenty of people have made them for plenty of languages and made them available. I just didn't manage to Google a Thai one so far.
    – hippietrail
    Commented Aug 30, 2013 at 18:02
  • 1
    Be careful with frequency, as sometimes infrequently used letters (and words) are nevertheless important. Raw written frequency should be attenuated to some degree. This is true in many cases, such as ฤ which is found in the word for "English" but otherwise quite rare.
    – jeffmcneill
    Commented May 19, 2014 at 9:38

1 Answer 1

Reset to default
5

I've downloaded the Thai Wikipedia dump of the 29/08/2013, converted its 82,200 articles into text files with wp2txt and counted the 195,667,724 characters with a small python script. wp2txt is supposed to remove the metadata, but the statistics can be skewed anyway, and since I don't know any Thai, I have no way to check whether the statistics below make any sense...

I've only kept the characters occurring with a probability higher than .01%, but I can decrease this threshold if you want. They are listed below, in decreasing frequency order, with their unicode names:

  • 0.0592 : - -:U+0020 SPACE
  • 0.0515 : -า-:U+0E32 THAI CHARACTER SARA AA
  • 0.0446 : -น-:U+0E19 THAI CHARACTER NO NU
  • 0.0428 : -ร-:U+0E23 THAI CHARACTER RO RUA
  • 0.0321 : -อ-:U+0E2D THAI CHARACTER O ANG
  • 0.0311 : -ก-:U+0E01 THAI CHARACTER KO KAI
  • 0.0306 : -เ-:U+0E40 THAI CHARACTER SARA E
  • 0.0293 : -ง-:U+0E07 THAI CHARACTER NGO NGU
  • 0.0282 : -่-:U+0E48 THAI CHARACTER MAI EK
  • 0.0258 : -ม-:U+0E21 THAI CHARACTER MO MA
  • 0.0255 : -ั-:U+0E31 THAI CHARACTER MAI HAN-AKAT
  • 0.0213 : -ย-:U+0E22 THAI CHARACTER YO YAK
  • 0.0211 : -ี-:U+0E35 THAI CHARACTER SARA II
  • 0.0211 : -้-:U+0E49 THAI CHARACTER MAI THO
  • 0.0206 : -ล-:U+0E25 THAI CHARACTER LO LING
  • 0.0195 : -ว-:U+0E27 THAI CHARACTER WO WAEN
  • 0.0180 : -ิ-:U+0E34 THAI CHARACTER SARA I
  • 0.0178 : -ท-:U+0E17 THAI CHARACTER THO THAHAN
  • 0.0178 : -ด-:U+0E14 THAI CHARACTER DO DEK
  • 0.0167 : -ส-:U+0E2A THAI CHARACTER SO SUA
  • 0.0158 : -ต-:U+0E15 THAI CHARACTER TO TAO
  • 0.0144 : -ะ-:U+0E30 THAI CHARACTER SARA A
  • 0.0139 : -บ-:U+0E1A THAI CHARACTER BO BAIMAI
  • 0.0139 : -ป-:U+0E1B THAI CHARACTER PO PLA
  • 0.0128 : -ค-:U+0E04 THAI CHARACTER KHO KHWAI
  • 0.0125 : -ห-:U+0E2B THAI CHARACTER HO HIP
  • 0.0114 : -แ-:U+0E41 THAI CHARACTER SARA AE
  • 0.0112 : -พ-:U+0E1E THAI CHARACTER PHO PHAN
  • 0.0108 : -จ-:U+0E08 THAI CHARACTER CHO CHAN

  • 0.0106 : -e-:U+0065 LATIN SMALL LETTER E

    • 1%

  • 0.0096 : -์-:U+0E4C THAI CHARACTER THANTHAKHAT

  • 0.0093 : -ช-:U+0E0A THAI CHARACTER CHO CHANG
  • 0.0092 : --:U+000A (end of line)
  • 0.0090 : -t-:U+0074 LATIN SMALL LETTER T
  • 0.0087 : -a-:U+0061 LATIN SMALL LETTER A
  • 0.0080 : -ข-:U+0E02 THAI CHARACTER KHO KHAI
  • 0.0080 : -ุ-:U+0E38 THAI CHARACTER SARA U
  • 0.0076 : -i-:U+0069 LATIN SMALL LETTER I
  • 0.0075 : -ใ-:U+0E43 THAI CHARACTER SARA AI MAIMUAN
  • 0.0073 : -ื-:U+0E37 THAI CHARACTER SARA UEE
  • 0.0071 : -ไ-:U+0E44 THAI CHARACTER SARA AI MAIMALAI
  • 0.0071 : -o-:U+006F LATIN SMALL LETTER O
  • 0.0069 : -n-:U+006E LATIN SMALL LETTER N
  • 0.0067 : -l-:U+006C LATIN SMALL LETTER L
  • 0.0066 : -โ-:U+0E42 THAI CHARACTER SARA O
  • 0.0065 : -r-:U+0072 LATIN SMALL LETTER R
  • 0.0065 : -็-:U+0E47 THAI CHARACTER MAITAIKHU
  • 0.0064 : -.-:U+002E FULL STOP
  • 0.0064 : -ู-:U+0E39 THAI CHARACTER SARA UU
  • 0.0059 : -ศ-:U+0E28 THAI CHARACTER SO SALA
  • 0.0054 : -ำ-:U+0E33 THAI CHARACTER SARA AM
  • 0.0054 : -s-:U+0073 LATIN SMALL LETTER S
  • 0.0049 : -2-:U+0032 DIGIT TWO
  • 0.0042 : -1-:U+0031 DIGIT ONE
  • 0.0041 : -p-:U+0070 LATIN SMALL LETTER P
  • 0.0041 : -0-:U+0030 DIGIT ZERO
  • 0.0037 : -ึ-:U+0E36 THAI CHARACTER SARA UE
  • 0.0036 : -ซ-:U+0E0B THAI CHARACTER SO SO
  • 0.0035 : -c-:U+0063 LATIN SMALL LETTER C
  • 0.0033 : -h-:U+0068 LATIN SMALL LETTER H
  • 0.0032 : -5-:U+0035 DIGIT FIVE
  • 0.0031 : -ถ-:U+0E16 THAI CHARACTER THO THUNG
  • 0.0031 : -)-:U+0029 RIGHT PARENTHESIS
  • 0.0031 : -(-:U+0028 LEFT PARENTHESIS
  • 0.0031 : -ธ-:U+0E18 THAI CHARACTER THO THONG
  • 0.0031 : -u-:U+0075 LATIN SMALL LETTER U
  • 0.0029 : -d-:U+0064 LATIN SMALL LETTER D
  • 0.0028 : -ภ-:U+0E20 THAI CHARACTER PHO SAMPHAO
  • 0.0028 : -ณ-:U+0E13 THAI CHARACTER NO NEN
  • 0.0028 : -ษ-:U+0E29 THAI CHARACTER SO RUSI
  • 0.0025 : -/-:U+002F SOLIDUS
  • 0.0025 : ---:U+002D HYPHEN-MINUS
  • 0.0025 : -m-:U+006D LATIN SMALL LETTER M
  • 0.0024 : -ผ-:U+0E1C THAI CHARACTER PHO PHUNG
  • 0.0024 : -|-:U+007C VERTICAL LINE
  • 0.0022 : -ญ-:U+0E0D THAI CHARACTER YO YING
  • 0.0022 : -g-:U+0067 LATIN SMALL LETTER G
  • 0.0022 : -,-:U+002C COMMA
  • 0.0021 : -4-:U+0034 DIGIT FOUR
  • 0.0020 : -3-:U+0033 DIGIT THREE
  • 0.0019 : -9-:U+0039 DIGIT NINE
  • 0.0018 : -[-:U+005B LEFT SQUARE BRACKET
  • 0.0018 : -]-:U+005D RIGHT SQUARE BRACKET
  • 0.0017 : -ฟ-:U+0E1F THAI CHARACTER FO FAN
  • 0.0017 : -w-:U+0077 LATIN SMALL LETTER W
  • 0.0017 : -=-:U+003D EQUALS SIGN
  • 0.0016 : -y-:U+0079 LATIN SMALL LETTER Y
  • 0.0015 : -8-:U+0038 DIGIT EIGHT
  • 0.0015 : -b-:U+0062 LATIN SMALL LETTER B
  • 0.0014 : -f-:U+0066 LATIN SMALL LETTER F
  • 0.0013 : -C-:U+0043 LATIN CAPITAL LETTER C
  • 0.0013 : -"-:U+0022 QUOTATION MARK
  • 0.0013 : -7-:U+0037 DIGIT SEVEN
  • 0.0013 : -6-:U+0036 DIGIT SIX
  • 0.0012 : -T-:U+0054 LATIN CAPITAL LETTER T
  • 0.0012 : -A-:U+0041 LATIN CAPITAL LETTER A
  • 0.0012 : -S-:U+0053 LATIN CAPITAL LETTER S
  • 0.0012 : -E-:U+0045 LATIN CAPITAL LETTER E
  • 0.0012 : -:-:U+003A COLON
  • 0.0011 : -R-:U+0052 LATIN CAPITAL LETTER R
  • 0.0010 : -ฐ-:U+0E10 THAI CHARACTER THO THAN

  • 0.0010 : -k-:U+006B LATIN SMALL LETTER K

    • .1 %

  • 0.0009 : -I-:U+0049 LATIN CAPITAL LETTER I

  • 0.0009 : -D-:U+0044 LATIN CAPITAL LETTER D
  • 0.0009 : -v-:U+0076 LATIN SMALL LETTER V
  • 0.0009 : -M-:U+004D LATIN CAPITAL LETTER M
  • 0.0008 : -B-:U+0042 LATIN CAPITAL LETTER B
  • 0.0008 : -P-:U+0050 LATIN CAPITAL LETTER P
  • 0.0008 : -ฮ-:U+0E2E THAI CHARACTER HO NOKHUK
  • 0.0006 : -ฉ-:U+0E09 THAI CHARACTER CHO CHING
  • 0.0006 : -N-:U+004E LATIN CAPITAL LETTER N
  • 0.0006 : -ฤ-:U+0E24 THAI CHARACTER RU
  • 0.0006 : -ๆ-:U+0E46 THAI CHARACTER MAIYAMOK
  • 0.0006 : -F-:U+0046 LATIN CAPITAL LETTER F
  • 0.0005 : -L-:U+004C LATIN CAPITAL LETTER L
  • 0.0005 : -ฝ-:U+0E1D THAI CHARACTER FO FA
  • 0.0005 : -H-:U+0048 LATIN CAPITAL LETTER H
  • 0.0005 : -G-:U+0047 LATIN CAPITAL LETTER G
  • 0.0005 : -W-:U+0057 LATIN CAPITAL LETTER W
  • 0.0004 : -O-:U+004F LATIN CAPITAL LETTER O
  • 0.0004 : -'-:U+0027 APOSTROPHE
  • 0.0004 : -ฎ-:U+0E0E THAI CHARACTER DO CHADA
  • 0.0004 : -ฏ-:U+0E0F THAI CHARACTER TO PATAK
  • 0.0004 : -J-:U+004A LATIN CAPITAL LETTER J
  • 0.0003 : -x-:U+0078 LATIN SMALL LETTER X
  • 0.0003 : -ฒ-:U+0E12 THAI CHARACTER THO PHUTHAO
  • 0.0003 : -K-:U+004B LATIN CAPITAL LETTER K
  • 0.0003 : -#-:U+0023 NUMBER SIGN
  • 0.0003 : -V-:U+0056 LATIN CAPITAL LETTER V
  • 0.0003 : -%-:U+0025 PERCENT SIGN
  • 0.0003 : -j-:U+006A LATIN SMALL LETTER J
  • 0.0002 : -U-:U+0055 LATIN CAPITAL LETTER U
  • 0.0002 : -๊-:U+0E4A THAI CHARACTER MAI TRI
  • 0.0002 : -z-:U+007A LATIN SMALL LETTER Z
  • 0.0002 : -–-:U+2013 EN DASH
  • 0.0002 : -ฑ-:U+0E11 THAI CHARACTER THO NANGMONTHO
  • 0.0002 : -ฯ-:U+0E2F THAI CHARACTER PAIYANNOI
  • 0.0002 : -ฬ-:U+0E2C THAI CHARACTER LO CHULA
  • 0.0002 : -”-:U+201D RIGHT DOUBLE QUOTATION MARK
  • 0.0002 : -“-:U+201C LEFT DOUBLE QUOTATION MARK
  • 0.0002 : -_-:U+005F LOW LINE
  • 0.0002 : -ฆ-:U+0E06 THAI CHARACTER KHO RAKHANG
  • 0.0002 : -Y-:U+0059 LATIN CAPITAL LETTER Y
  • 0.0001 : -&-:U+0026 AMPERSAND
  • 0.0001 : -;-:U+003B SEMICOLON
  • 0.0001 : -๋-:U+0E4B THAI CHARACTER MAI CHATTAWA
Improve this answer
8
  • You didn't have to do that! But thanks (-: I have done this for word frequency quite a lot and always find there are lots of overrepresented words in Wikipedia that are part of its "interface" or "structure". Such as "Edit", "External links", "citation needed", etc. This is going to skew the character frequency too, though probably to a lesser extent at my guess. This list is certainly useful while something better in unavailable.
    – hippietrail
    Commented Aug 31, 2013 at 16:09
  • When I read your question, I realized I was waiting for an opportunity to do that ;-) And wp2txt is supposed to remove a big part of the wikipedia specific part. I've run it with the -dtaesg options in order to minimize this skew. However, some structure is still visible in the text files, but it is mainly in ASCII characters, and therefore easily separated from the 'real data'.
    – Frédéric Grosshans
    Commented Aug 31, 2013 at 16:42
  • Aha I glossed over "wp2txt" when reading your answer, rather than reading up on what it actually tries to do. In my code I have some logic that starts with the HTML of the page without the "framing" header, footer, and sidebars, then converts to plain Unicode text. It makes no attempt to remove "inline interface" such as edit links, standard section headings, or template-generated text. Boilerpipe might do a more thorough job, but I don't know how either really compare to wp2txt.
    – hippietrail
    Commented Aug 31, 2013 at 16:53
  • 1
    I'm now in Laos facing the same situation with Lao. It is related and has a related script, but easier spelling. If you feel like it let me know and I will post a similar question and you can answer.
    – hippietrail
    Commented Sep 7, 2013 at 16:28
  • 1
    @hippietrail : Characters in lao wikipedia, in decreasing frequency order : - -າ -ນ -່ -ງ -ກ -ດ -ັ -ະ -້ -ເ -ອ -​ -ມ -ວ -ສ -ລ -ີ -ທ -ົ -ບ -ປ -ແ -ຫ -ຍ -ິ -ຕ -e -ຄ -ພ -ຊ -ື -(return) -a -ຂ -t -i -ຈ -n -o -. -ໃ -r -ໄ -ູ -l -ຸ -s -ຳ -ໍ -, -ຮ -ໂ -ຖ -0 -ຽ -c -1 -h -d -u -p -ຜ -ຼ -2 -ຢ -( -) -ຶ -m -­ -- -ໜ -g -ຣ -f -9 -| -y -: -ໝ -b -= -3 -5 -C -4 -/ -[ -] -S -8 -7 -E -T -v -w -R -6 -I -ຟ -A -ໆ - -k -P -… -D -B -M -N -L -" -ຝ -G -H -W -F -x -O -໌ -U -J -# -; -K -z -า -V -' -* -ร -” -“ -น -— -j -% -q -Y -& -– -é -X -ั -ก -ง
    – Frédéric Grosshans
    Commented Sep 8, 2013 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.