0

Is there any way to have a good heuristic study on the frequency of special characters and uppercase ?

Would studying a freely available corpus (around 15m words) generates a good estimation?

Edit: I would like to study the frequency of occurrence in English of special characters ( e.g. !@#$%^&*()_+?~-/[]{} ) compared to letters

where I would like to construct a frequency table for all characters ( e.g. saying the frequency of occurance of @ is 0.05% of all texts )

Improve this question
9
  • What are special characters? What language and subject matter do you want their frequencies for?
    – brass tacks
    May 10, 2015 at 20:53
  • for example: !@#$%^&*()?":/;-~ and the language is english. Frequency of occurrence compared to other characters ( say compared to letters an such ) where I can say that @ has a frequency of 0.05% and so on
    – Tt22
    May 11, 2015 at 3:55
  • What is the application (what do you need this for)?
    – brass tacks
    May 11, 2015 at 3:58
  • We would like to expand on the frequency studies of letters to include uppercase/special characters/numbers
    – Tt22
    May 11, 2015 at 4:01
  • So you just want to know for general written English? It will be different depending on the subject matter; for example, the frequency will probably be higher in math papers or programming guides and lower in novels.
    – brass tacks
    May 11, 2015 at 4:03

1 Answer 1

Reset to default
2

Regarding corpus size: If you need meaningful results for the less frequent characters, you'll need a larger corpus, such as the ones at http://corpus.byu.edu.

Their web interface accepts queries consisting of punctuation. A sample query is .|,|:|;|!|?|'|"|@|#|$|%|^|&|*|(|)|-|~, which for the Contemporary American English, Global Web-based English and Wikipedia corpora produces these frequencies:

COCA               GloWbE             Wikipedia
 1 , 23849941       1 , 83847760       1 , 109960441
 2 . 21304861       2 . 80894692       2 '  79264498
 3 "  7008881       3 " 17572733       3 .  73633968
 4 )  1694276       4 )  8668155       4 #  28560000
 5 (  1688162       5 (  8337855       5 (  25894803
 6 ?  1620720       6 ?  7853684       6 )  25851855
 7 :  1360298       7 :  6323921       7 *  18572182
 8 '   990783       8 '  6179597       8 :  10649543
 9 ;   782692       9 !  4029929       9 ;   5709259
10 #   545703      10 ;  2511484      10 -   4912002
11 -   392700      11 -  2228750      11 !   3248850
12 !   368622      12 *   137176      12 ?    378872
13 $   133473      13 @    66127      13 @     20287
14 &   109713      14 %    27920      14 %     16899
15 *    81934      15 $    23881      15 $     10455
16 @    74050      16 #    22256
17 %     1761

(The Wikipedia corpus has " encoded as '' and uses # as a special character.)

However, if you need to study digits, capitals, or the symbols [ ] / | < >, you need to download the corpora and do your own analysis.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.