Questions tagged [frequency]

How often a linguistic element occurs relative to other linguistic elements.

Filter by
Sorted by
Tagged with
0
votes
0answers
77 views

Does an alphabet with the uniform letter frequency distribution exist?

A language employs some kind of alphabet for writing. One could naïvely expect that each letter bears the same amount of entropy. But in reality that is not the case. For example in English each ...
5
votes
2answers
103 views

Are Word Frequencies Cross-Lingual?

If someone tried to invent a code where they simply replaced every English word with another word, the code could be cracked (given a large enough sample) by comparing frequencies of English words to ...
5
votes
1answer
65 views

Word length as a function of word frequency?

I was wondering if any studies have been done in corpus linguistics which look at word length (number of phonemes per word) as a function of the word's frequency of use? Thanks! EDIT: Never mind, ...
1
vote
3answers
79 views

Words with the hightest frequency that are not Collocates of each other (inside a text corpora)

Is there a way to obtain a: List, that has 2 words per row Each 2 words are not collocates (or collocations) of each other (ie: are not found within the same phrase inside the entire text corpus, or ...
3
votes
2answers
91 views

Which free resource to find word frequency?

My hypothesis is that math jargon is more aliening than other academic fields, and I want to concrete that by an example of the average frequency of the word in these two definitions of "...
0
votes
0answers
25 views

How to determine word frequency based on demographic? (Does a public library of word frequencies exist by demographic)

I'm not sure "sign post" questions are appropriate however that is what this is. I am looking for a tool, a means of answering questions such as "Word X is used more by demographic Y than demographic ...
3
votes
1answer
71 views

Do more closely related languages, have more similar Zipfian distributions?

"Zipf's law" is just a pretentious way of saying that many types of data, in various sciences, fit certain kinds of power law distribution. E.g. in linguistics, for a corpus of English word frequency ...
0
votes
0answers
218 views

Complete List of English Compound Words?

I am looking for a complete list of English compound words (or a frequency list of English compound words). Ideally, I'd prefer a list that lists the compound word and also its constituent words (...
4
votes
0answers
356 views

Is there a phoneme distribution graph for multiple European languages?

I am doing some research on a manuscript which I need to identify the language. My hypothesis is that it is written in phoneme by someone who does not understand the language. Spoken aloud, one that ...
1
vote
1answer
45 views

Speech signal analysis: What can be seen in the signal?

I know that the only speech phenomena one can see in a speech signal are those that arise from articulator movement and which have features that aid perception (like energy as particular freqs+times ...
1
vote
0answers
31 views

Error of word frequency [closed]

I have created a database of lyrics with some 10000 songs and did a word frequency analysis to find the most common words. Next, i sorted the corpus according to the publication date and measured the ...
3
votes
1answer
212 views

Are there any good theories explaining the language aspect of the Zipf Mystery?

The Zipf mystery: In (presumably all from what I have read) natural languages, the number of times any specific word shows up in the language as a whole, or any particular piece is approximately ...
3
votes
0answers
135 views

Distribution of the set of meanings of a given word, in a corpus

I am looking to know about the distribution of word senses, of a particular word in a corpus. How often a word sense will occur out of all the occurrences of that word with any meaning. For example: ...
0
votes
0answers
79 views

How common is onomatopoeia in English?

My intuition is that onomatopoeia is quite rare and that the letters and sounds of most words are meaningless (e.g. if "chicken" and "fish" were switched all along, no one would think "That word doesn'...
1
vote
1answer
172 views

Collocations frequency dictionary of English

I am looking for a top list of the most frequent collocations in English. The desirable size of the list is more than 1000 items. The best that I found so far is this collocations list by ...
1
vote
0answers
158 views

The usage frequency of individual English words

I'm very new to NLP and I'm sure this question is a rookie one. In NLP for English language, how do we tell if a word is commonly used, from one that is not commonly used? What is the simplest measure?...
0
votes
1answer
392 views

Standard Deviation Of Fundamental Frequency Of Males and Females [closed]

I know that it fundamental frequency is usually considered to be normally distributed, and that females have a higher variance in their distribution than males. What is the variance of the fundamental ...
0
votes
3answers
2k views

most frequent syllables

I am looking for a list of the most frequent syllables of all languages spoken on earth, sorted by frequency. I found such lists for english and for german, but I want to get a list across all ...
1
vote
2answers
212 views

Frequency of phonemes consisting of two phones in world languages

I want to create nice name for international service. My idea is based on the fact, the word which consisting of phonemes that have high frequency in the native language, sounds good. So, the same ...
0
votes
4answers
116 views

What's a good technique to understand the differences between the time and frequency domains?

I understand that phonetic analysis can be performed in various domains, among which: time, frequency, cepstral, and spectral. In the time domain, one studies e.g. Rate, energy, and duration. In the ...
0
votes
1answer
448 views

Special Characters and Uppercase Frequency?

Is there any way to have a good heuristic study on the frequency of special characters and uppercase ? Would studying a freely available corpus (around 15m words) generates a good estimation? Edit: ...
0
votes
2answers
93 views

Expected Frequency

I am reading an article where the authors mention something they call the expected frequency of a word in a corpus without explaining or giving a definition of what this means. The article in ...
2
votes
1answer
289 views

How to determine difficulty of a word if its frequency in a corpus is known?

What is known in general about the relationship between word difficulty and word frequency? That is, for a given word, suppose I know the frequency with which it (or its word stem) is encountered in ...
2
votes
1answer
143 views

What is considered the smallest possible sample size for word frequency lists used in FL instruction?

I've been engaged in a conversation on another site pertaining to frequency analysis, particularly in relation to the 1966 work Buntús Gaeilge, Colmán Ó Huallacháin; Ireland. Department of ...
3
votes
2answers
308 views

Probabilities for 2-grams are higher than 1-grams in arpa file produced by kenlm

I'm using the 1 billion word language corpus to build a model with 1 and 2-grams. When using the lmplz program that comes with kenlm, I noticed that the arpa file seems to have higher probabilities ...
3
votes
2answers
249 views

Word and/or syllable frequency data for Lao

I've returned my language focus to Lao now that my travels through Asia have finished and I'm back home. There are not as many or as high quality resources available for Lao as for many other ...
3
votes
1answer
889 views

Comparing frequencies of two corpora

I want to compare two corpora. One with 15000 tokens and the other with 5000 tokens. Can I normalise the word frequencies per 100000 words? Or I have to normalise based on words per million?
4
votes
0answers
507 views

Mandarin Chinese syllable and tone frequency (not character frequency)

There's plenty of good resources on Chinese character frequency available. But I'm wondering about syllable frequency independent of characters, and also tone frequency both separate and in ...
14
votes
4answers
4k views

Are there natural languages that do not obey Zipf's law?

Is there a natural language which is known not to follow Zipf's law? I'm interested to see if it's really universal. This is what Zipf's law states: Zipf's law states that given some corpus of ...
0
votes
2answers
171 views

Software to form a word frequency list for a Spanish book

I've actually found one called "textstat", but it doesn't recognize different word forms of one word. So are there programs that do detect comía, come, cómelo, comiera as forms of comer?
1
vote
0answers
68 views

Connection between word usage frequency and number of synonyms for the word

At a first glance it seems to be obvious/expectable (at least to some extent) that the more frequently a word is used in language the more synonyms it should have. Can somebody point out whether this ...
5
votes
2answers
3k views

How to establish word frequencies from a corpus?

I have a English language corpus consisting of 100 files of different sizes and containing a total of 7 million words. What is a good approach in order to establish the frequency of a given word from ...
1
vote
1answer
859 views

Looking for Thai letter-frequency resource

I'm currently in Thailand and a bit frustrated with the difficult writing system here. Normally after a week in a country I can learn the writing system to a functional degree (except Han characters ...
5
votes
1answer
739 views

Words and phrases more likely in everyday speech

I'm processing a large corpus for a given language. I've noticed that for certain sections of the corpus, I can go many tens of thousands of words without the word "me" being mentioned. These are ...
13
votes
5answers
25k views

How many “words” do I need to learn?

I am interested in learning a second language. To do this I have created a list of the 1000 most common words and phrases for a given language. I've also established sentences which contains each of ...
1
vote
0answers
350 views

What mechanism of language can be used to explain competitive synonyms similar to high-frequency word?

Suppose we have two competitive synonyms A and B. We found out that during the historical changes of the language, A gains dominant position. We found out the fact above via a statistical survey of ...
3
votes
1answer
1k views

Word frequency list for agglutinative languages like Swahili?

I'm an intermediate Swahili speaker looking to substantially increase the size of my vocabulary over the next couple weeks. However, I want to optimize my learning curve by focusing on the first 2000 ...
9
votes
3answers
3k views

Where to find frequency list of English words from newspapers, books and magazines?

I'd like to download a frequency list of English words that are most common in newspapers, books and magazines. It should contain at least 50.000 words. Is it possible to download such a list from ...
1
vote
0answers
212 views

Searching for a Spanish word list for statistical analysis [closed]

I need a Spanish word list, as simple as that. The more complete it is, the better, it should contain as many words as to be a statistically relevant sample. It can be in any format: XML, MS Excel, ....
3
votes
1answer
354 views

What is the idea of ngram viewers working case-sensitive? [closed]

On German Language & Usage we recently had a question about the English word schmuck, which is a false friend of the German word Schmuck. Out of curiosity I tried the capitalized form vs the non ...
6
votes
1answer
299 views

Are there tools similar Google ngram, which recognise conjugated or declined word forms?

I just found out, that with respect to the German language, where prefixes of verbs are separated from the verb and posed behind the object, I have no chance to find out, if a verb with a prefix is ...
3
votes
1answer
3k views

What are the most commonly used Chinese syllables?

There are about 1200 or so unique (includes all tones) Mandarin Chinese syllables, according to some source I read a while back based off the Unihan database. For my applications I'm limited to using ...