Questions tagged [corpora]

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

Filter by
Sorted by
Tagged with
0 votes
1 answer
17 views

Comparing Text Translations by Conducting Corpus Analysis

Provided one has three pieces of text called - “Text A”, “Text B” and “Text C”. Text A is the original text. Text B and Text C are translations of Text A (Text A, being initially written in another ...
5 votes
3 answers
951 views

How can I find the frequency for the Russian word "чем-нибудь"?

I've tried searching for the word "чем-нибудь"on the 20,000 word list I found on Wikipedia. You can link directly to it here. For the sake of thoroughness, I even searched the larger 50,...
1 vote
1 answer
66 views

Can someone share with me an article that studies the decline of by-phrases in the passive?

Can someone share with me an article that studies the decline of by-phrases in the passive? Preferably a corpus-based study, but this is not that relevant
2 votes
0 answers
43 views

Quantitative methodology for contrastive pragmatics in corpus-based settings

I am interested in literature regarding methodology that could be relevant for quantitative research into differences in pragmatic meaning between two 'equivalent' concepts in two languages (in other ...
1 vote
0 answers
52 views

What is a good multi language translation system for individual parts of speech such as verbs, nouns, adjectives?

I am developing a language learning tool in Python that generates dual-language books intended to be read as audio books. The system should work by giving single word translations after every ...
0 votes
0 answers
34 views

Minimum collocation frequency for corpora?

I have a corpus which contains newspaper articles about trans men and trans women. I want to carry out collocational analysis on this corpus, focusing on verbs. However, I'm not sure what my minimum ...
2 votes
1 answer
80 views

Are there any set guidelines for the lifespan of a (spoken) corpus?

Are there any set guidelines on how long a (spoken) corpus stays relevant / is reliable to use? The only major Dutch spoken corpus is from 2004, and I was wondering if using this corpus would cause a ...
3 votes
1 answer
56 views

Bilingual corpora work from the late 1980s

Does anyone know if the paper by Warwick and Russell (often cited as below) exists? I have looked on the EURALEX website and the authors are not listed as speakers in the 4th Conference. Maybe the ...
0 votes
1 answer
47 views

Are alveolo-palatal consonants more likely to be followed by high vowels, whereas retroflex consonants are more likely followed by low vowels?

It seems to me that high voles like i would more naturally follow alveolo-palatal consonants because the need to "spread the lips" (in the popular description of the latter) seem to more ...
0 votes
0 answers
30 views

Vocabulary Rate for Spanish

I am searching for a resource in Spanish that contains the Spanish Vocabulary Rate, classifying words for their usage level, similar to this Italian one: Nuovo Vocabolario di Base della lingua ...
4 votes
2 answers
463 views

I am looking for an Arabic ngram corpus

I am working in a project where i need to use an ngram model. So, i want to know if an Arabic ngram corpus exist. I have tried to find a corpus but all my researches failed. I know that for languages ...
0 votes
1 answer
180 views

Can someone explain to me the Zipf–Mandelbrot law?

I really can't understand. It's about linguistics and I can't understand anything because there are mathematical formulas in it that I can't understand at all. Can anyone explain this with ...
19 votes
3 answers
9k views

Does anyone know of text message corpora?

I am looking for a large corpus of text messages. By large, I am hoping to have at least 15,000 text messages in my sample. I am fine with combining several smaller corpora into a larger corpus as I ...
0 votes
1 answer
83 views

What is the frequency of color words in English compared to German?

I am writing a term paper about the role of adjectives and gender marking in efficient communication about English and German respectively. And I was wondering if there is any way/website/tool to ...
2 votes
2 answers
131 views

register variation: unbalanced corpus sample

I'm doing a small pilot study on evidentiality in a language I work on. I'm looking at two evidential markers, -mi that is said to mark the source of knowledge as personal, and -shi that marks ...
1 vote
1 answer
94 views

Has anyone ever ranked the prevalence of phones by number of speakers worldwide?

I'm interested in knowing the most-used and least-used phones worldwide. According to Wikipedia, the IPA charts about 140 pulmonic consonants, 80 non-pulmonic consonants, 30 co-articulated consonants, ...
1 vote
1 answer
40 views

Large parallel corpus of English and Farsi text

hope you are doing well. I am looking for a large parallel corpus of English and Farsi text, professionally written or edited. I would be more than happy to have your suggestions. Thank you all.
0 votes
1 answer
37 views

Where can I find a corpus of sentence with the resulting emotion in french?

I'm working on a sentiment analysis program and I need to validate it using a corpus of sentence with the linked emotions. I need to perform this task on a french corpus and I can't find any on the ...
1 vote
1 answer
90 views

Is there any free, single-word wordlist with CLAWS7 part-of-speech tags?

I am looking for a wordlist which includes single-word entries and their part-of-speech tags in UCREL CLAWS7 format. The list must be comprised of scientifically collected data, for example words ...
3 votes
7 answers
10k views

Sounds that humans make with their mouths

I'm interested in a list of sounds that humans make with their mouths. I'm not looking for speech sounds sounds that take a non-average skill (such as beatboxing) sounds that cannot be made with the ...
9 votes
7 answers
5k views

Biggest freely available English corpus?

Any help on finding the biggest freely available English corpus that can be used on research? So far I have found OANC with 15 M words.
0 votes
1 answer
57 views

Where can I get the complete list of word entries present in Merriam Webster's dictionary?

I can't find any list, github repo, etc to find the total list of entries defined in merriam webster. Any ideas?
11 votes
2 answers
609 views

How to best clean a large historical corpus ridden with OCR errors

Overview: I have a very large corpus of historical news papers (17th-20th cent.). The word count is about 20 bln. It's raw OCR-ed data in txt-files of about 150 GB. One newspaper issue per file (some ...
1 vote
0 answers
41 views

Corpus of Taboo card game?

Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
3 votes
1 answer
101 views

How order of the syntax tree is formally/strictly proven?

An example from https://en.wikipedia.org/wiki/Concrete_syntax_tree : tree of "John hit the ball." . The tree branches are joined in this order: {John {hit {the ball}}}. Ie, "the" ...
4 votes
2 answers
408 views

How to interpret this form of Heaps' Law?

Heaps' Law basically is an empirical function that says the number of distinct words you'll find in a document grows as a function to the length of the document. The equation given in the Wikipedia ...
2 votes
2 answers
416 views

Where to find/purchase an offline (downloadable) Arabic corpus

I'm developing an application to teach Quran. I need some textual/lingual analysis based on a common and general corpus. I know about King Abdulaziz City for Science and Technology (KACST) Arabic ...
5 votes
3 answers
656 views

Are there tools to build domain specific corpus? [closed]

I would like to build a domain specific corpus. By this I mean a corpus from a specific knowledge area, for instance Law, Psychiatry, or Klingon language. I am currently using a Wikipedia corpus, and ...
6 votes
2 answers
1k views

Is there any corpus for technical English? (E.g., computers, IT, modern technology)

Alternatively, do any larger corpora have categories that pertain to technology, computer science, etc.? Or collections of English documentation and/or manuals related to modern devices, interfaces, ...
5 votes
1 answer
493 views

Where online can I find a list of all the Hapax Legomena in the Hebrew Bible?

I am looking for a list online of all Hapax Legomena in the Hebrew Bible in text format that I can copy and paste into MS Word for further studies. I need something that's free and is not in a picture ...
1 vote
2 answers
80 views

Is it possible to test statistical significance of difference between two distance/similarity scores?

I have a corpus consisting of tweets by men and another corpus of female tweets. I was thinking of using a word embeddings approach (e.g., glove and fasttext) and examine the cosine similarity between ...
0 votes
1 answer
133 views

Switchboard queries with NXT (the TEXT function)

The NXT query language includes the query function TEXT, which should return the textual content of the variable it applies to. In theory, Switchboard should support this. The file README.SWBD-QUERIES....
2 votes
2 answers
3k views

Most commonly used words across languages?

I'm doing my own (very small) corpus analysis, including a word frequency list and would like to compare the top, say, 30 words with other such lists. I know that there are things like the GSL/NGSL, ...
2 votes
0 answers
50 views

Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?

If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language? If yes, do point ...
7 votes
1 answer
838 views

How can I calculate if the difference between two word frequencies in one corpus is significant?

I want to study orthographical variants, for example: Can firefighter and fire-fighter be considered orthographical variants (i.e. the difference in frequency in a corpus is not statistically ...
1 vote
2 answers
140 views

How can I search a corpus for easy sentences?

I would like to search a large corpus for example sentences which contain exclusively or a high proportion of words from the N most common words in some frequency dictionary (the language of the ...
0 votes
2 answers
47 views

Are there corpora which tag phrases by discourse- or conversational-function?

More generally, imagine a corpus which could identify the conversational function of a sentence, such as, "Have a nice day!", in narrow categories like, "ending a conversation", or broad ones like "...
0 votes
3 answers
134 views

Which is the most recommended corpus for training a POS Tagger?

The tagger is for English and I will use the universal dependency tag set. I am looking for a corpus that best represents nowadays English.
1 vote
1 answer
60 views

Praat - window size larger than vowel

I'm doing an acoustic-phonetic analysis of read-speech in an American English corpus. If we want to set a window/frame size (25 - 30 ms) to cover the majority of vowels in the corpus, but some vowels (...
2 votes
0 answers
49 views

Word alignment for Switchboard Dialogue Acts Corpus

I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here. I can see how it is possible to match up every utterance ...
2 votes
0 answers
129 views

Is the source list of "Oxford English Corpus" public available?

The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
4 votes
3 answers
2k views

Word frequency list for agglutinative languages like Swahili?

I'm an intermediate Swahili speaker looking to substantially increase the size of my vocabulary over the next couple weeks. However, I want to optimize my learning curve by focusing on the first 2000 ...
2 votes
0 answers
97 views

What word category do twink, adonis, muse etc. belong to? [closed]

Terms such as candy, cutie, honey, princess, diamond, queen(?), stud and bunny are terms of endearment (these terms are often used in a relationship to show affection and may also be used, with ...
1 vote
1 answer
95 views

Spoken Arabic dictionary/corpus?

Is there an existing corpus or dictionary for different spoken Arabic dialects? I'd like to look up a word (in English or MSA or in a spoken dialect) and find where this word or phrase is spoken (and ...
6 votes
1 answer
68 views

Corpus of English sentences written by non-native speakers?

I am working on a hobby project which tends to help the non-native speakers figure out if the sentences they write are good enough and sounds like written by English native speakers. Any relevant info ...
1 vote
1 answer
68 views

Comparing corpora

So first, a disclaimer: I'm less than beginner level in everything that has to do with programming. I'm analysing the occurrences of English code-switching/translanguaging in a particular Polish-...
3 votes
0 answers
52 views

Searching for an English Whats-app Corpus

I am searching an English Whats-app Corpus in order to analyse a linguistic phenomena. I had some difficulties to find one and maybe some of you can help me out. It is only for corpus driven study and ...
1 vote
0 answers
46 views

Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?

I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
6 votes
1 answer
229 views

How to make a Corpus? (selecting a good structure)

I'm making a Corpus. I have searched for good practices on how to build it. I've downloaded some Corpus to check their internal structure and the extension file, but I didn't find many. So, I am lost ...
1 vote
4 answers
225 views

Is it possible to build thesaurus automatically?

Is it possible to build thesaurus by simply analyzing lots of text, or it's obligatory should be handmade? Also a question: is there free-to-download&use English language thesaurus?

1
2 3 4 5