Questions tagged [corpora]
A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.
230
questions
3
votes
0
answers
61
views
Corpus studies on the frequency of subject questions in English
Are there any corpus studies which show the relative frequency of different types of interrogative main clauses in English, in particular the relative frequency of subject questions (which do not ...
- 31
1
vote
0
answers
28
views
Dealing with words having token-frequency 1 when calculating Phonotactic Probability
NOTE
This is a repost of a question on the Mathematics StackExchange Board. It has been modified to fit this board :)
I am open to suggestions
Explanation
I am trying to calculate phonotactic ...
- 111
0
votes
1
answer
49
views
Corpus Linguistics Major [closed]
I want to use corpus in my MA research, but I have a large problem; how can I analyze the data? Do I use software corpora or corpus such as BNC? I know it is according to my research question, but ...
- 19
-2
votes
1
answer
62
views
How could you study “cultural differences” using corpora? [closed]
Anyone who’s traveled has come upon a sense of there being some “cultural differences”, but my mind is always seeking to systematize things, and I’ve wondered if there is a better way to collect tons ...
- 371
10
votes
2
answers
449
views
Are linguistic corpora under threat from AI-generated text?
As many will be aware, there has been a lot of concern about the spread of AI generated text posted on the Stack Exchange network, leading to moderators on several sites standing down. See here and ...
- 3,962
1
vote
2
answers
84
views
Should I use normalized frequencies or raw frequencies when conducting Log-likelihood test to compare two corpora with differing text lenghts?
Analyzing multiword expressions and I want to compare two corpora of similar sizes, but these corpora differ in average text length. For example average text length in Corpus A. is 300 while it is 600 ...
0
votes
0
answers
30
views
YCOE Old English Corpus
I need some help in figuring out how to use the YCOE Old English Corpus. Specifically, I need to write a query that will allow me find all the occurences of temese ("Thames") as well as find ...
- 1
2
votes
0
answers
69
views
COCA corpus: Is it possible to display a list search result as parts of speech?
I want to know what parts of speech (and in what frequency) follow a word or phrase. If I do a list search, for example "for the purpose of *", the result is a list of the actual words in ...
1
vote
1
answer
72
views
Where to find English POS words?
Given a POS (part of speech), tag, and dependency for words, where can I find words that have the same POS, tag, and dependency?
I only need a list of words by (pos,tag) and do not need code for this (...
- 21
0
votes
1
answer
54
views
I need help with searching a corpus for a complex sentence
I am trying to formulate a search request that will show me example sentences for sentences that contain the phrase "In the beginning" or "At the beginning" followed by the word &...
- 11
0
votes
0
answers
33
views
Is there any mathematically studied ontology space?
I am considering “ontology generation”. I have not yet read the specifics of these techniques.
Still, the point must surely be to identify some kind of cooccurrences / associations between words. ...
- 371
0
votes
0
answers
39
views
Where can I find a free/cheap corpus of the 20,000 most frequently used words in the English language? [duplicate]
There are a bunch of large corpora containing millions or billions of words.
My needs are simpler: I only need the 20K most used words. Preferably, these words should be most used in the general ...
0
votes
1
answer
31
views
Comparing Text Translations by Conducting Corpus Analysis
Provided one has three pieces of text called - “Text A”, “Text B” and “Text C”.
Text A is the original text.
Text B and Text C are translations of Text A (Text A, being initially written in another ...
1
vote
1
answer
67
views
Can someone share with me an article that studies the decline of by-phrases in the passive?
Can someone share with me an article that studies the decline of by-phrases in the passive? Preferably a corpus-based study, but this is not that relevant
user36925
3
votes
0
answers
46
views
Quantitative methodology for contrastive pragmatics in corpus-based settings
I am interested in literature regarding methodology that could be relevant for quantitative research into differences in pragmatic meaning between two 'equivalent' concepts in two languages (in other ...
1
vote
0
answers
56
views
What is a good multi language translation system for individual parts of speech such as verbs, nouns, adjectives?
I am developing a language learning tool in Python that generates dual-language books intended to be read as audio books.
The system should work by giving single word translations after every ...
- 11
0
votes
1
answer
66
views
Are alveolo-palatal consonants more likely to be followed by high vowels, whereas retroflex consonants are more likely followed by low vowels?
It seems to me that high voles like i would more naturally follow alveolo-palatal consonants because the need to "spread the lips" (in the popular description of the latter) seem to more ...
- 175
5
votes
3
answers
1k
views
How can I find the frequency for the Russian word "чем-нибудь"?
I've tried searching for the word "чем-нибудь"on the 20,000 word list I found on Wikipedia. You can link directly to it here. For the sake of thoroughness, I even searched the larger 50,...
- 313
0
votes
1
answer
239
views
Can someone explain to me the Zipf–Mandelbrot law?
I really can't understand. It's about linguistics and I can't understand anything because there are mathematical formulas in it that I can't understand at all. Can anyone explain this with ...
- 35
3
votes
1
answer
60
views
Bilingual corpora work from the late 1980s
Does anyone know if the paper by Warwick and Russell (often cited as below) exists? I have looked on the EURALEX website and the authors are not listed as speakers in the 4th Conference. Maybe the ...
- 171
2
votes
1
answer
119
views
Has anyone ever ranked the prevalence of phones by number of speakers worldwide?
I'm interested in knowing the most-used and least-used phones worldwide. According to Wikipedia, the IPA charts about 140 pulmonic consonants, 80 non-pulmonic consonants, 30 co-articulated consonants, ...
- 121
0
votes
1
answer
102
views
What is the frequency of color words in English compared to German?
I am writing a term paper about the role of adjectives and gender marking in efficient communication about English and German respectively.
And I was wondering if there is any way/website/tool to ...
1
vote
1
answer
48
views
Large parallel corpus of English and Farsi text
hope you are doing well.
I am looking for a large parallel corpus of English and Farsi text, professionally written or edited. I would be more than happy to have your suggestions.
Thank you all.
- 13
0
votes
1
answer
43
views
Where can I find a corpus of sentence with the resulting emotion in french?
I'm working on a sentiment analysis program and I need to validate it using a corpus of sentence with the linked emotions.
I need to perform this task on a french corpus and I can't find any on the ...
- 101
2
votes
1
answer
82
views
Are there any set guidelines for the lifespan of a (spoken) corpus?
Are there any set guidelines on how long a (spoken) corpus stays relevant / is reliable to use? The only major Dutch spoken corpus is from 2004, and I was wondering if using this corpus would cause a ...
0
votes
1
answer
181
views
Where can I get the complete list of word entries present in Merriam Webster's dictionary?
I can't find any list, github repo, etc to find the total list of entries defined in merriam webster. Any ideas?
- 109
11
votes
2
answers
726
views
How to best clean a large historical corpus ridden with OCR errors
Overview:
I have a very large corpus of historical news papers (17th-20th cent.). The word count is about 20 bln. It's raw OCR-ed data in txt-files of about 150 GB. One newspaper issue per file (some ...
- 231
1
vote
0
answers
42
views
Corpus of Taboo card game?
Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
- 576
3
votes
1
answer
128
views
How order of the syntax tree is formally/strictly proven?
An example from https://en.wikipedia.org/wiki/Concrete_syntax_tree : tree of "John hit the ball." .
The tree branches are joined in this order:
{John {hit {the ball}}}.
Ie, "the" ...
- 131
5
votes
1
answer
885
views
Where online can I find a list of all the Hapax Legomena in the Hebrew Bible?
I am looking for a list online of all Hapax Legomena in the Hebrew Bible in text format that I can copy and paste into MS Word for further studies.
I need something that's free and is not in a picture ...
1
vote
2
answers
85
views
Is it possible to test statistical significance of difference between two distance/similarity scores?
I have a corpus consisting of tweets by men and another corpus of female tweets. I was thinking of using a word embeddings approach (e.g., glove and fasttext) and examine the cosine similarity between ...
- 121
2
votes
0
answers
61
views
Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?
If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language?
If yes, do point ...
- 337
1
vote
1
answer
110
views
Praat - window size larger than vowel
I'm doing an acoustic-phonetic analysis of read-speech in an American English corpus.
If we want to set a window/frame size (25 - 30 ms) to cover the majority of vowels in the corpus, but some vowels (...
- 11
2
votes
0
answers
77
views
Word alignment for Switchboard Dialogue Acts Corpus
I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here.
I can see how it is possible to match up every utterance ...
- 576
2
votes
0
answers
210
views
Is the source list of "Oxford English Corpus" public available?
The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
- 121
1
vote
2
answers
211
views
How can I search a corpus for easy sentences?
I would like to search a large corpus for example sentences which contain exclusively or a high proportion of words from the N most common words in some frequency dictionary (the language of the ...
- 179
1
vote
0
answers
123
views
What word category do twink, adonis, muse etc. belong to? [closed]
Terms such as candy, cutie, honey, princess, diamond, queen(?), stud and bunny are terms of endearment (these terms are often used in a relationship to show affection and may also be used, with ...
- 111
1
vote
1
answer
117
views
Spoken Arabic dictionary/corpus?
Is there an existing corpus or dictionary for different spoken Arabic dialects? I'd like to look up a word (in English or MSA or in a spoken dialect) and find where this word or phrase is spoken (and ...
- 191
6
votes
1
answer
81
views
Corpus of English sentences written by non-native speakers?
I am working on a hobby project which tends to help the non-native speakers figure out if the sentences they write are good enough and sounds like written by English native speakers. Any relevant info ...
- 163
0
votes
3
answers
208
views
Which is the most recommended corpus for training a POS Tagger?
The tagger is for English and I will use the universal dependency tag set. I am looking for a corpus that best represents nowadays English.
- 11
1
vote
1
answer
73
views
Comparing corpora
So first, a disclaimer: I'm less than beginner level in everything that has to do with programming.
I'm analysing the occurrences of English code-switching/translanguaging in a particular Polish-...
- 11
3
votes
1
answer
103
views
Searching for an English Whats-app Corpus
I am searching an English Whats-app Corpus in order to analyse a linguistic phenomena. I had some difficulties to find one and maybe some of you can help me out. It is only for corpus driven study and ...
- 82
2
votes
0
answers
48
views
Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?
I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
- 21
6
votes
1
answer
357
views
How to make a Corpus? (selecting a good structure)
I'm making a Corpus.
I have searched for good practices on how to build it.
I've downloaded some Corpus to check their internal structure and the extension file, but I didn't find many.
So, I am lost ...
- 63
1
vote
1
answer
52
views
Suggestion for software to analyze data from social media- please advise
I am searching urgent for a software that I can use to analyse corpora. In short, I would like to have a look at collocations, ellipsis, frequencies, concordances etc. I also want to search for themes ...
- 11
6
votes
1
answer
131
views
What are the conventions for corpus usage?
Suppose I'm working with a language that doesn't have much data available; the largest corpus around is a translation of the Bible, which is free to read online, but is presumably still copyrighted by ...
- 64.1k
1
vote
1
answer
59
views
Is there a comprehensive list of corpora of all the languages (maybe even dialects) of the world?
Is there a comprehensive list of corpora of all the languages of the world?
I've been searching on wikipedia eg. https://en.wikipedia.org/wiki/Czech_National_Corpus. If I scroll to the bottom, I do ...
- 337
2
votes
1
answer
199
views
Does anyone have a link to the American Local News Corpus V1.0?
I found a paper http://www.cs.jhu.edu/~anni/papers/alnc_lrec14.pdf that has made this epic text corpus of over 1 billion words available somewhere, but I can't find it anywhere online.
The paper says ...
- 191
0
votes
1
answer
39
views
Need an online freely available Anaphorically Annotated Corpus of English Language for Identification of Discourse Units
I'm in need of a freely available corpus of English language (which may contain text of any genre) but has to be annotated anaphoriacally i-e. Anaphora resolution should have been performed already.
...
4
votes
0
answers
194
views
Is there an english news corpus available to download for between 1900 and 201X (free or low cost)
I'm attempting a word embedding analysis (think underlying meaning and implications, but computational) of certain keywords through time in the English language, but I am having some difficulty ...
- 191