# Which free resource to find word frequency?

My hypothesis is that math jargon is more aliening than other academic fields, and I want to concrete that by an example of the average frequency of the word in these two definitions of "representation" in politic and math:

• the activity of making citizens’ voices, opinions, and perspectives "present" in public policy making processes
• the homomorphism from an abstract algebraic object on a vector space

Since I don't have an interest in purchasing a whole corpus, is there a free resource for me to find the word frequency? COCA only provides 5000 random words, Google Ngram Viewer is only about books, and Wikitionary doesn't provide a unique file to download. I guess I can find out with the googlefu:

``````site:https://en.wiktionary.org/wiki/Wiktionary:Frequency_lists/PG/2005/08 "keyword"
``````

but is there a better way to do that?

• The relationship between your hypothesis and the experiment you seem to be devising to test it seems muddy at best. How would you disambiguate the meanings in a corpus, and how then does whatever result you get support or contradict your hypothesis? Jun 25, 2018 at 6:57
• I think to measure the familiarity of a whole field one needs to use a whole corpus on that field, not just one example, but I can't find field-depended corpora. I think the frequency or prevalence of the word is a measure on the familiarity. Does that answer your question? Jun 25, 2018 at 7:21
• I don't think that's a convincing line of reasoning, no. But the main stumbling block regardless of this line of reasoning is establishing which meaning is the intended one for each sample manually; how do you plan to pull that off? Jun 25, 2018 at 7:25
• Do you mean by how I know whether "representation" is used as a math term or politic term? To a layman they will just use the normal sense to understand the technical sense, so asking them what the formal definitions mean they can only wild guess that, which I think wouldn't be much derivative than the lexical definition. But if we present the two formal definitions, if one of them makes them feel interesting for the rigorousness and gain more insight on the word, and one of them is like Greek, then this is the difference? Jun 25, 2018 at 7:43

Word frequency is only a proxy for word knowledge. For the English language, there are data available on word prevalence, i.e., on how many people know a certain word. You can find Measures of word prevalence for 61,800 English words by Marc Brysbaert, Paweł Mandera, Samantha McCormick, and Emmanuel Keuleers in the link given.

If what you really want is Wiktionary's frequency list, you can get that by taking the various Wiktionary pages linked from the overview page and splicing them together.

However, be careful which corpus you're using. The idea of a "universal" corpus is generally a myth: comparing the frequencies of these words in TV scripts versus Project Gutenberg versus Twitter scrapes versus the local mathematics library will give quite different results. You'll need to decide what exactly you're going to take as representative of English as a whole.

And on the topic of your proposed usage, separate from choosing a word list itself…

You also need to be wary of the definitions you're using. You've chosen a very common political word that everyone knows, and a fairly obscure mathematical term that someone in e.g. statistics would have no interaction whatsoever with. If you want a fair comparison between math jargon and political science jargon, you'll need to choose a word that's not used in the same sense outside political science. For example:

Constitutional autochthony is the process of asserting constitutional nationalism from an external legal or political power.

(From Wikipedia.)

Or, choose a form of the mathematical definition which is designed for outside audiences:

A collection of objects A, B, C… is said to represent a collection of objects 1, 2, 3… if there is some consistent analogy you can make between A and 1, and between B and 2, and so on.

• I've read closely at the Project Gutenberg, and realize that it only counts words from books like 100 years ago, so it's better to stick with Google Ngram? As for the method, I think it's better to just measure the average frequency of the jargon used in each academic field. To do that I need to have the corpus of each field. Do you know where I can get (or better a paper has done this analysis)? Jun 19, 2018 at 4:21