Questions tagged [corpora]

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

Filter by
Sorted by
Tagged with
3 votes
0 answers
61 views

Corpus studies on the frequency of subject questions in English

Are there any corpus studies which show the relative frequency of different types of interrogative main clauses in English, in particular the relative frequency of subject questions (which do not ...
user43197's user avatar
1 vote
0 answers
28 views

Dealing with words having token-frequency 1 when calculating Phonotactic Probability

NOTE This is a repost of a question on the Mathematics StackExchange Board. It has been modified to fit this board :) I am open to suggestions Explanation I am trying to calculate phonotactic ...
corvusMidnight's user avatar
0 votes
1 answer
49 views

Corpus Linguistics Major [closed]

I want to use corpus in my MA research, but I have a large problem; how can I analyze the data? Do I use software corpora or corpus such as BNC? I know it is according to my research question, but ...
Yous Gaber's user avatar
-2 votes
1 answer
62 views

How could you study “cultural differences” using corpora? [closed]

Anyone who’s traveled has come upon a sense of there being some “cultural differences”, but my mind is always seeking to systematize things, and I’ve wondered if there is a better way to collect tons ...
Julius H.'s user avatar
  • 371
10 votes
2 answers
449 views

Are linguistic corpora under threat from AI-generated text?

As many will be aware, there has been a lot of concern about the spread of AI generated text posted on the Stack Exchange network, leading to moderators on several sites standing down. See here and ...
Araucaria - him's user avatar
1 vote
2 answers
84 views

Should I use normalized frequencies or raw frequencies when conducting Log-likelihood test to compare two corpora with differing text lenghts?

Analyzing multiword expressions and I want to compare two corpora of similar sizes, but these corpora differ in average text length. For example average text length in Corpus A. is 300 while it is 600 ...
An Average Linguist's user avatar
0 votes
0 answers
30 views

YCOE Old English Corpus

I need some help in figuring out how to use the YCOE Old English Corpus. Specifically, I need to write a query that will allow me find all the occurences of temese ("Thames") as well as find ...
JCBC's user avatar
  • 1
2 votes
0 answers
69 views

COCA corpus: Is it possible to display a list search result as parts of speech?

I want to know what parts of speech (and in what frequency) follow a word or phrase. If I do a list search, for example "for the purpose of *", the result is a list of the actual words in ...
Seymour Azzes's user avatar
1 vote
1 answer
72 views

Where to find English POS words?

Given a POS (part of speech), tag, and dependency for words, where can I find words that have the same POS, tag, and dependency? I only need a list of words by (pos,tag) and do not need code for this (...
rlgekdcgc's user avatar
0 votes
1 answer
54 views

I need help with searching a corpus for a complex sentence

I am trying to formulate a search request that will show me example sentences for sentences that contain the phrase "In the beginning" or "At the beginning" followed by the word &...
Mara's user avatar
  • 11
0 votes
0 answers
33 views

Is there any mathematically studied ontology space?

I am considering “ontology generation”. I have not yet read the specifics of these techniques. Still, the point must surely be to identify some kind of cooccurrences / associations between words. ...
Julius H.'s user avatar
  • 371
0 votes
0 answers
39 views

Where can I find a free/cheap corpus of the 20,000 most frequently used words in the English language? [duplicate]

There are a bunch of large corpora containing millions or billions of words. My needs are simpler: I only need the 20K most used words. Preferably, these words should be most used in the general ...
Montana Burr Freelancing's user avatar
0 votes
1 answer
31 views

Comparing Text Translations by Conducting Corpus Analysis

Provided one has three pieces of text called - “Text A”, “Text B” and “Text C”. Text A is the original text. Text B and Text C are translations of Text A (Text A, being initially written in another ...
JavaApprentice's user avatar
1 vote
1 answer
67 views

Can someone share with me an article that studies the decline of by-phrases in the passive?

Can someone share with me an article that studies the decline of by-phrases in the passive? Preferably a corpus-based study, but this is not that relevant
user avatar
3 votes
0 answers
46 views

Quantitative methodology for contrastive pragmatics in corpus-based settings

I am interested in literature regarding methodology that could be relevant for quantitative research into differences in pragmatic meaning between two 'equivalent' concepts in two languages (in other ...
Damiaan Reijnaers's user avatar
1 vote
0 answers
56 views

What is a good multi language translation system for individual parts of speech such as verbs, nouns, adjectives?

I am developing a language learning tool in Python that generates dual-language books intended to be read as audio books. The system should work by giving single word translations after every ...
Holden's user avatar
  • 11
0 votes
1 answer
66 views

Are alveolo-palatal consonants more likely to be followed by high vowels, whereas retroflex consonants are more likely followed by low vowels?

It seems to me that high voles like i would more naturally follow alveolo-palatal consonants because the need to "spread the lips" (in the popular description of the latter) seem to more ...
Fizz's user avatar
  • 175
5 votes
3 answers
1k views

How can I find the frequency for the Russian word "чем-нибудь"?

I've tried searching for the word "чем-нибудь"on the 20,000 word list I found on Wikipedia. You can link directly to it here. For the sake of thoroughness, I even searched the larger 50,...
LISA's user avatar
  • 313
0 votes
1 answer
239 views

Can someone explain to me the Zipf–Mandelbrot law?

I really can't understand. It's about linguistics and I can't understand anything because there are mathematical formulas in it that I can't understand at all. Can anyone explain this with ...
Sami Bülbül's user avatar
3 votes
1 answer
60 views

Bilingual corpora work from the late 1980s

Does anyone know if the paper by Warwick and Russell (often cited as below) exists? I have looked on the EURALEX website and the authors are not listed as speakers in the 4th Conference. Maybe the ...
Hbar's user avatar
  • 171
2 votes
1 answer
119 views

Has anyone ever ranked the prevalence of phones by number of speakers worldwide?

I'm interested in knowing the most-used and least-used phones worldwide. According to Wikipedia, the IPA charts about 140 pulmonic consonants, 80 non-pulmonic consonants, 30 co-articulated consonants, ...
Lawton's user avatar
  • 121
0 votes
1 answer
102 views

What is the frequency of color words in English compared to German?

I am writing a term paper about the role of adjectives and gender marking in efficient communication about English and German respectively. And I was wondering if there is any way/website/tool to ...
Fee Arnold's user avatar
1 vote
1 answer
48 views

Large parallel corpus of English and Farsi text

hope you are doing well. I am looking for a large parallel corpus of English and Farsi text, professionally written or edited. I would be more than happy to have your suggestions. Thank you all.
MOHAMMAD's user avatar
0 votes
1 answer
43 views

Where can I find a corpus of sentence with the resulting emotion in french?

I'm working on a sentiment analysis program and I need to validate it using a corpus of sentence with the linked emotions. I need to perform this task on a french corpus and I can't find any on the ...
J.erome's user avatar
  • 101
2 votes
1 answer
82 views

Are there any set guidelines for the lifespan of a (spoken) corpus?

Are there any set guidelines on how long a (spoken) corpus stays relevant / is reliable to use? The only major Dutch spoken corpus is from 2004, and I was wondering if using this corpus would cause a ...
Artos van stel's user avatar
0 votes
1 answer
181 views

Where can I get the complete list of word entries present in Merriam Webster's dictionary?

I can't find any list, github repo, etc to find the total list of entries defined in merriam webster. Any ideas?
Alireza's user avatar
  • 109
11 votes
2 answers
726 views

How to best clean a large historical corpus ridden with OCR errors

Overview: I have a very large corpus of historical news papers (17th-20th cent.). The word count is about 20 bln. It's raw OCR-ed data in txt-files of about 150 GB. One newspaper issue per file (some ...
Mat's user avatar
  • 231
1 vote
0 answers
42 views

Corpus of Taboo card game?

Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
Adam_G's user avatar
  • 576
3 votes
1 answer
128 views

How order of the syntax tree is formally/strictly proven?

An example from https://en.wikipedia.org/wiki/Concrete_syntax_tree : tree of "John hit the ball." . The tree branches are joined in this order: {John {hit {the ball}}}. Ie, "the" ...
qdinar's user avatar
  • 131
5 votes
1 answer
885 views

Where online can I find a list of all the Hapax Legomena in the Hebrew Bible?

I am looking for a list online of all Hapax Legomena in the Hebrew Bible in text format that I can copy and paste into MS Word for further studies. I need something that's free and is not in a picture ...
Reb Chaim HaQoton's user avatar
1 vote
2 answers
85 views

Is it possible to test statistical significance of difference between two distance/similarity scores?

I have a corpus consisting of tweets by men and another corpus of female tweets. I was thinking of using a word embeddings approach (e.g., glove and fasttext) and examine the cosine similarity between ...
SanMelkote's user avatar
2 votes
0 answers
61 views

Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?

If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language? If yes, do point ...
forgodsakehold's user avatar
1 vote
1 answer
110 views

Praat - window size larger than vowel

I'm doing an acoustic-phonetic analysis of read-speech in an American English corpus. If we want to set a window/frame size (25 - 30 ms) to cover the majority of vowels in the corpus, but some vowels (...
AyushiP's user avatar
  • 11
2 votes
0 answers
77 views

Word alignment for Switchboard Dialogue Acts Corpus

I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here. I can see how it is possible to match up every utterance ...
Adam_G's user avatar
  • 576
2 votes
0 answers
210 views

Is the source list of "Oxford English Corpus" public available?

The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
brennn's user avatar
  • 121
1 vote
2 answers
211 views

How can I search a corpus for easy sentences?

I would like to search a large corpus for example sentences which contain exclusively or a high proportion of words from the N most common words in some frequency dictionary (the language of the ...
julkarham's user avatar
  • 179
1 vote
0 answers
123 views

What word category do twink, adonis, muse etc. belong to? [closed]

Terms such as candy, cutie, honey, princess, diamond, queen(?), stud and bunny are terms of endearment (these terms are often used in a relationship to show affection and may also be used, with ...
Karl L.'s user avatar
  • 111
1 vote
1 answer
117 views

Spoken Arabic dictionary/corpus?

Is there an existing corpus or dictionary for different spoken Arabic dialects? I'd like to look up a word (in English or MSA or in a spoken dialect) and find where this word or phrase is spoken (and ...
VirtualValentin's user avatar
6 votes
1 answer
81 views

Corpus of English sentences written by non-native speakers?

I am working on a hobby project which tends to help the non-native speakers figure out if the sentences they write are good enough and sounds like written by English native speakers. Any relevant info ...
user3768495's user avatar
0 votes
3 answers
208 views

Which is the most recommended corpus for training a POS Tagger?

The tagger is for English and I will use the universal dependency tag set. I am looking for a corpus that best represents nowadays English.
Yadiel Velez's user avatar
1 vote
1 answer
73 views

Comparing corpora

So first, a disclaimer: I'm less than beginner level in everything that has to do with programming. I'm analysing the occurrences of English code-switching/translanguaging in a particular Polish-...
Joanneboron's user avatar
3 votes
1 answer
103 views

Searching for an English Whats-app Corpus

I am searching an English Whats-app Corpus in order to analyse a linguistic phenomena. I had some difficulties to find one and maybe some of you can help me out. It is only for corpus driven study and ...
Jannik's user avatar
  • 82
2 votes
0 answers
48 views

Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?

I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
user27273's user avatar
6 votes
1 answer
357 views

How to make a Corpus? (selecting a good structure)

I'm making a Corpus. I have searched for good practices on how to build it. I've downloaded some Corpus to check their internal structure and the extension file, but I didn't find many. So, I am lost ...
XOREZ's user avatar
  • 63
1 vote
1 answer
52 views

Suggestion for software to analyze data from social media- please advise

I am searching urgent for a software that I can use to analyse corpora. In short, I would like to have a look at collocations, ellipsis, frequencies, concordances etc. I also want to search for themes ...
Cathe's user avatar
  • 11
6 votes
1 answer
131 views

What are the conventions for corpus usage?

Suppose I'm working with a language that doesn't have much data available; the largest corpus around is a translation of the Bible, which is free to read online, but is presumably still copyrighted by ...
Draconis's user avatar
  • 64.1k
1 vote
1 answer
59 views

Is there a comprehensive list of corpora of all the languages (maybe even dialects) of the world?

Is there a comprehensive list of corpora of all the languages of the world? I've been searching on wikipedia eg. https://en.wikipedia.org/wiki/Czech_National_Corpus. If I scroll to the bottom, I do ...
forgodsakehold's user avatar
2 votes
1 answer
199 views

Does anyone have a link to the American Local News Corpus V1.0?

I found a paper http://www.cs.jhu.edu/~anni/papers/alnc_lrec14.pdf that has made this epic text corpus of over 1 billion words available somewhere, but I can't find it anywhere online. The paper says ...
Joe B's user avatar
  • 191
0 votes
1 answer
39 views

Need an online freely available Anaphorically Annotated Corpus of English Language for Identification of Discourse Units

I'm in need of a freely available corpus of English language (which may contain text of any genre) but has to be annotated anaphoriacally i-e. Anaphora resolution should have been performed already. ...
Muhammad Aatif's user avatar
4 votes
0 answers
194 views

Is there an english news corpus available to download for between 1900 and 201X (free or low cost)

I'm attempting a word embedding analysis (think underlying meaning and implications, but computational) of certain keywords through time in the English language, but I am having some difficulty ...
Joe B's user avatar
  • 191

1
2 3 4 5