Skip to main content

Questions tagged [corpora]

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

Filter by
Sorted by
Tagged with
0 votes
1 answer
40 views

Ideal number of randomized concordancing lines

I want to perform randomized concordancing on a news corpus with 807 mil. tokens from 2 mil. documents from 2011-2022. Is there any ideal number of concordancing in order to be able to make a ...
pindakazen's user avatar
0 votes
0 answers
17 views

Quantative/Statistical comparison between unequal corpora

I have created a corpus of 400.000 words, consisting exclusively of governmental administrative documents. I am focusing on the usage of rare words and i want to prove that my corpus has increased ...
vigilantius22's user avatar
0 votes
0 answers
23 views

Is plain orthographic transciption of speech recordings adequate for corpus creation?

NOTE: I am not a linguist, but my interest is in the Natural Language Processing (NLP) field. I am doing a research which is focused only on gathering new or distinct words on a recently described ...
Rod Maniego's user avatar
0 votes
0 answers
21 views

Are open-ended questions allowed for an interview focused on corpus creation?

NOTE: I am not a linguist, but my interest is in the Natural Language Processing (NLP) field. I am doing a research which is focused only on gathering new or distinct words on a recently described ...
Rod Maniego's user avatar
1 vote
0 answers
40 views

"data cleaning" of tatoeba sentence file

I'm trying to automate Anki deck creation for my language learning. To this end, I downloaded a file with sentences from Tatoeba. My goal is to calculate the frequency ranking of each word in every ...
Z98HefcKk9bS's user avatar
-1 votes
2 answers
68 views

Do I need to lemmatize my corpus when performing collocation analysis on inflective languages?

Do I need to lemmatize my corpus when I am going to perform a collocational analysis? Several language, including English, are to a certain extent inflective. These inflections sometimes carries ...
pindakazen's user avatar
0 votes
0 answers
55 views

Using semantic preference as method for critical linguistics research?

I am intending to conduct a critical linguistics research using corpus linguistics as a method, particularly, the analysis of semantic preference analysis to word-pair patterns (henceforth collocates)....
pindakazen's user avatar
1 vote
0 answers
55 views

What differentiates the usage of MI and MI3 in collocation analysis?

In Brezina (2018), it is stated that to measure collocation strength, one may use the Mutual Information (MI) to test the collocational strength of a given collocation. MI is naturally more sensitive ...
pindakazen's user avatar
0 votes
1 answer
34 views

What does the word "company" mean in the context of collocation analysis in corpus linguistics?

I am currently reading Brezina (2018) on Statistics in Corpus Linguistics. I have a hard time trying to decipher what the author means of "company" in this sentence: Dice and log Dice, and ...
pindakazen's user avatar
1 vote
1 answer
33 views

Why use log per-million count in this analysis of Brevity Law using corpora?

I am curious about corpus linguistics, and especially in this case how corpora is used to prove the Brevity Law, also known as Zip'f law of abbreviation. Simply put, this theorem postulates that the ...
pindakazen's user avatar
3 votes
0 answers
66 views

Corpus studies on the frequency of subject questions in English

Are there any corpus studies which show the relative frequency of different types of interrogative main clauses in English, in particular the relative frequency of subject questions (which do not ...
user43197's user avatar
2 votes
0 answers
32 views

Dealing with words having token-frequency 1 when calculating Phonotactic Probability

NOTE This is a repost of a question on the Mathematics StackExchange Board. It has been modified to fit this board :) I am open to suggestions Explanation I am trying to calculate phonotactic ...
corvusMidnight's user avatar
0 votes
1 answer
54 views

Corpus Linguistics Major [closed]

I want to use corpus in my MA research, but I have a large problem; how can I analyze the data? Do I use software corpora or corpus such as BNC? I know it is according to my research question, but ...
Yous Gaber's user avatar
-2 votes
1 answer
65 views

How could you study “cultural differences” using corpora? [closed]

Anyone who’s traveled has come upon a sense of there being some “cultural differences”, but my mind is always seeking to systematize things, and I’ve wondered if there is a better way to collect tons ...
Julius Hamilton's user avatar
10 votes
2 answers
470 views

Are linguistic corpora under threat from AI-generated text?

As many will be aware, there has been a lot of concern about the spread of AI generated text posted on the Stack Exchange network, leading to moderators on several sites standing down. See here and ...
Araucaria - him's user avatar
1 vote
2 answers
158 views

Should I use normalized frequencies or raw frequencies when conducting Log-likelihood test to compare two corpora with differing text lenghts?

Analyzing multiword expressions and I want to compare two corpora of similar sizes, but these corpora differ in average text length. For example average text length in Corpus A. is 300 while it is 600 ...
An Average Linguist's user avatar
2 votes
0 answers
79 views

COCA corpus: Is it possible to display a list search result as parts of speech?

I want to know what parts of speech (and in what frequency) follow a word or phrase. If I do a list search, for example "for the purpose of *", the result is a list of the actual words in ...
Seymour Azzes's user avatar
1 vote
1 answer
78 views

Where to find English POS words?

Given a POS (part of speech), tag, and dependency for words, where can I find words that have the same POS, tag, and dependency? I only need a list of words by (pos,tag) and do not need code for this (...
rlgekdcgc's user avatar
0 votes
1 answer
54 views

I need help with searching a corpus for a complex sentence

I am trying to formulate a search request that will show me example sentences for sentences that contain the phrase "In the beginning" or "At the beginning" followed by the word &...
Mara's user avatar
  • 11
0 votes
0 answers
36 views

Is there any mathematically studied ontology space?

I am considering “ontology generation”. I have not yet read the specifics of these techniques. Still, the point must surely be to identify some kind of cooccurrences / associations between words. ...
Julius Hamilton's user avatar
0 votes
0 answers
40 views

Where can I find a free/cheap corpus of the 20,000 most frequently used words in the English language? [duplicate]

There are a bunch of large corpora containing millions or billions of words. My needs are simpler: I only need the 20K most used words. Preferably, these words should be most used in the general ...
Montana Burr Freelancing's user avatar
0 votes
1 answer
33 views

Comparing Text Translations by Conducting Corpus Analysis

Provided one has three pieces of text called - “Text A”, “Text B” and “Text C”. Text A is the original text. Text B and Text C are translations of Text A (Text A, being initially written in another ...
JavaApprentice's user avatar
1 vote
1 answer
69 views

Can someone share with me an article that studies the decline of by-phrases in the passive?

Can someone share with me an article that studies the decline of by-phrases in the passive? Preferably a corpus-based study, but this is not that relevant
user avatar
3 votes
0 answers
46 views

Quantitative methodology for contrastive pragmatics in corpus-based settings

I am interested in literature regarding methodology that could be relevant for quantitative research into differences in pragmatic meaning between two 'equivalent' concepts in two languages (in other ...
Damiaan Reijnaers's user avatar
1 vote
0 answers
58 views

What is a good multi language translation system for individual parts of speech such as verbs, nouns, adjectives?

I am developing a language learning tool in Python that generates dual-language books intended to be read as audio books. The system should work by giving single word translations after every ...
Holden's user avatar
  • 11
0 votes
1 answer
73 views

Are alveolo-palatal consonants more likely to be followed by high vowels, whereas retroflex consonants are more likely followed by low vowels?

It seems to me that high voles like i would more naturally follow alveolo-palatal consonants because the need to "spread the lips" (in the popular description of the latter) seem to more ...
against very long user names's user avatar
5 votes
3 answers
1k views

How can I find the frequency for the Russian word "чем-нибудь"?

I've tried searching for the word "чем-нибудь"on the 20,000 word list I found on Wikipedia. You can link directly to it here. For the sake of thoroughness, I even searched the larger 50,...
LISA's user avatar
  • 313
0 votes
1 answer
304 views

Can someone explain to me the Zipf–Mandelbrot law?

I really can't understand. It's about linguistics and I can't understand anything because there are mathematical formulas in it that I can't understand at all. Can anyone explain this with ...
Sami Bülbül's user avatar
3 votes
1 answer
61 views

Bilingual corpora work from the late 1980s

Does anyone know if the paper by Warwick and Russell (often cited as below) exists? I have looked on the EURALEX website and the authors are not listed as speakers in the 4th Conference. Maybe the ...
Hbar's user avatar
  • 171
2 votes
1 answer
127 views

Has anyone ever ranked the prevalence of phones by number of speakers worldwide?

I'm interested in knowing the most-used and least-used phones worldwide. According to Wikipedia, the IPA charts about 140 pulmonic consonants, 80 non-pulmonic consonants, 30 co-articulated consonants, ...
Lawton's user avatar
  • 121
0 votes
1 answer
106 views

What is the frequency of color words in English compared to German?

I am writing a term paper about the role of adjectives and gender marking in efficient communication about English and German respectively. And I was wondering if there is any way/website/tool to ...
Fee Arnold's user avatar
1 vote
1 answer
50 views

Large parallel corpus of English and Farsi text

hope you are doing well. I am looking for a large parallel corpus of English and Farsi text, professionally written or edited. I would be more than happy to have your suggestions. Thank you all.
MOHAMMAD's user avatar
0 votes
1 answer
44 views

Where can I find a corpus of sentence with the resulting emotion in french?

I'm working on a sentiment analysis program and I need to validate it using a corpus of sentence with the linked emotions. I need to perform this task on a french corpus and I can't find any on the ...
J.erome's user avatar
  • 101
2 votes
1 answer
83 views

Are there any set guidelines for the lifespan of a (spoken) corpus?

Are there any set guidelines on how long a (spoken) corpus stays relevant / is reliable to use? The only major Dutch spoken corpus is from 2004, and I was wondering if using this corpus would cause a ...
Artos van stel's user avatar
0 votes
1 answer
294 views

Where can I get the complete list of word entries present in Merriam Webster's dictionary?

I can't find any list, github repo, etc to find the total list of entries defined in merriam webster. Any ideas?
Alireza's user avatar
  • 109
11 votes
2 answers
777 views

How to best clean a large historical corpus ridden with OCR errors

Overview: I have a very large corpus of historical news papers (17th-20th cent.). The word count is about 20 bln. It's raw OCR-ed data in txt-files of about 150 GB. One newspaper issue per file (some ...
Mat's user avatar
  • 231
1 vote
0 answers
42 views

Corpus of Taboo card game?

Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
Adam_G's user avatar
  • 576
3 votes
1 answer
145 views

How order of the syntax tree is formally/strictly proven?

An example from https://en.wikipedia.org/wiki/Concrete_syntax_tree : tree of "John hit the ball." . The tree branches are joined in this order: {John {hit {the ball}}}. Ie, "the" ...
qdinar's user avatar
  • 131
5 votes
1 answer
1k views

Where online can I find a list of all the Hapax Legomena in the Hebrew Bible?

I am looking for a list online of all Hapax Legomena in the Hebrew Bible in text format that I can copy and paste into MS Word for further studies. I need something that's free and is not in a picture ...
Reb Chaim HaQoton's user avatar
1 vote
2 answers
86 views

Is it possible to test statistical significance of difference between two distance/similarity scores?

I have a corpus consisting of tweets by men and another corpus of female tweets. I was thinking of using a word embeddings approach (e.g., glove and fasttext) and examine the cosine similarity between ...
SanMelkote's user avatar
2 votes
0 answers
63 views

Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?

If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language? If yes, do point ...
forgodsakehold's user avatar
1 vote
1 answer
135 views

Praat - window size larger than vowel

I'm doing an acoustic-phonetic analysis of read-speech in an American English corpus. If we want to set a window/frame size (25 - 30 ms) to cover the majority of vowels in the corpus, but some vowels (...
AyushiP's user avatar
  • 11
2 votes
0 answers
85 views

Word alignment for Switchboard Dialogue Acts Corpus

I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here. I can see how it is possible to match up every utterance ...
Adam_G's user avatar
  • 576
2 votes
0 answers
218 views

Is the source list of "Oxford English Corpus" public available?

The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
brennn's user avatar
  • 121
1 vote
2 answers
232 views

How can I search a corpus for easy sentences?

I would like to search a large corpus for example sentences which contain exclusively or a high proportion of words from the N most common words in some frequency dictionary (the language of the ...
julkarham's user avatar
  • 179
1 vote
0 answers
124 views

What word category do twink, adonis, muse etc. belong to? [closed]

Terms such as candy, cutie, honey, princess, diamond, queen(?), stud and bunny are terms of endearment (these terms are often used in a relationship to show affection and may also be used, with ...
Karl L.'s user avatar
  • 111
1 vote
1 answer
130 views

Spoken Arabic dictionary/corpus?

Is there an existing corpus or dictionary for different spoken Arabic dialects? I'd like to look up a word (in English or MSA or in a spoken dialect) and find where this word or phrase is spoken (and ...
VirtualValentin's user avatar
6 votes
1 answer
86 views

Corpus of English sentences written by non-native speakers?

I am working on a hobby project which tends to help the non-native speakers figure out if the sentences they write are good enough and sounds like written by English native speakers. Any relevant info ...
user3768495's user avatar
0 votes
3 answers
231 views

Which is the most recommended corpus for training a POS Tagger?

The tagger is for English and I will use the universal dependency tag set. I am looking for a corpus that best represents nowadays English.
Yadiel Velez's user avatar
1 vote
1 answer
78 views

Comparing corpora

So first, a disclaimer: I'm less than beginner level in everything that has to do with programming. I'm analysing the occurrences of English code-switching/translanguaging in a particular Polish-...
Joanneboron's user avatar

1
2 3 4 5