Questions tagged [corpora]
A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.
225
questions
2
votes
0
answers
49
views
COCA corpus: Is it possible to display a list search result as parts of speech?
I want to know what parts of speech (and in what frequency) follow a word or phrase. If I do a list search, for example "for the purpose of *", the result is a list of the actual words in ...
1
vote
1
answer
64
views
Where to find English POS words?
Given a POS (part of speech), tag, and dependency for words, where can I find words that have the same POS, tag, and dependency?
I only need a list of words by (pos,tag) and do not need code for this (...
- 21
0
votes
1
answer
45
views
I need help with searching a corpus for a complex sentence
I am trying to formulate a search request that will show me example sentences for sentences that contain the phrase "In the beginning" or "At the beginning" followed by the word &...
- 11
0
votes
0
answers
26
views
Is there any mathematically studied ontology space?
I am considering “ontology generation”. I have not yet read the specifics of these techniques.
Still, the point must surely be to identify some kind of cooccurrences / associations between words. ...
- 782
0
votes
0
answers
37
views
Where can I find a free/cheap corpus of the 20,000 most frequently used words in the English language? [duplicate]
There are a bunch of large corpora containing millions or billions of words.
My needs are simpler: I only need the 20K most used words. Preferably, these words should be most used in the general ...
0
votes
0
answers
15
views
Corpus linguistics: finding a BCCWJ sentence by ID from Universal Dependencies
I am using an online Universal Dependencies search (http://universal.grew.fr/?corpus=UD_Japanese-BCCWJ@2.10) to look at the prevalence of certain grammatical structures in languages including Japanese....
0
votes
1
answer
23
views
Comparing Text Translations by Conducting Corpus Analysis
Provided one has three pieces of text called - “Text A”, “Text B” and “Text C”.
Text A is the original text.
Text B and Text C are translations of Text A (Text A, being initially written in another ...
1
vote
1
answer
67
views
Can someone share with me an article that studies the decline of by-phrases in the passive?
Can someone share with me an article that studies the decline of by-phrases in the passive? Preferably a corpus-based study, but this is not that relevant
user36925
2
votes
0
answers
44
views
Quantitative methodology for contrastive pragmatics in corpus-based settings
I am interested in literature regarding methodology that could be relevant for quantitative research into differences in pragmatic meaning between two 'equivalent' concepts in two languages (in other ...
1
vote
0
answers
55
views
What is a good multi language translation system for individual parts of speech such as verbs, nouns, adjectives?
I am developing a language learning tool in Python that generates dual-language books intended to be read as audio books.
The system should work by giving single word translations after every ...
- 11
0
votes
1
answer
58
views
Are alveolo-palatal consonants more likely to be followed by high vowels, whereas retroflex consonants are more likely followed by low vowels?
It seems to me that high voles like i would more naturally follow alveolo-palatal consonants because the need to "spread the lips" (in the popular description of the latter) seem to more ...
- 175
5
votes
3
answers
978
views
How can I find the frequency for the Russian word "чем-нибудь"?
I've tried searching for the word "чем-нибудь"on the 20,000 word list I found on Wikipedia. You can link directly to it here. For the sake of thoroughness, I even searched the larger 50,...
- 313
0
votes
1
answer
209
views
Can someone explain to me the Zipf–Mandelbrot law?
I really can't understand. It's about linguistics and I can't understand anything because there are mathematical formulas in it that I can't understand at all. Can anyone explain this with ...
- 37
3
votes
1
answer
58
views
Bilingual corpora work from the late 1980s
Does anyone know if the paper by Warwick and Russell (often cited as below) exists? I have looked on the EURALEX website and the authors are not listed as speakers in the 4th Conference. Maybe the ...
- 171
1
vote
1
answer
108
views
Has anyone ever ranked the prevalence of phones by number of speakers worldwide?
I'm interested in knowing the most-used and least-used phones worldwide. According to Wikipedia, the IPA charts about 140 pulmonic consonants, 80 non-pulmonic consonants, 30 co-articulated consonants, ...
- 111
0
votes
1
answer
97
views
What is the frequency of color words in English compared to German?
I am writing a term paper about the role of adjectives and gender marking in efficient communication about English and German respectively.
And I was wondering if there is any way/website/tool to ...
1
vote
1
answer
45
views
Large parallel corpus of English and Farsi text
hope you are doing well.
I am looking for a large parallel corpus of English and Farsi text, professionally written or edited. I would be more than happy to have your suggestions.
Thank you all.
- 13
0
votes
1
answer
41
views
Where can I find a corpus of sentence with the resulting emotion in french?
I'm working on a sentiment analysis program and I need to validate it using a corpus of sentence with the linked emotions.
I need to perform this task on a french corpus and I can't find any on the ...
- 101
2
votes
1
answer
82
views
Are there any set guidelines for the lifespan of a (spoken) corpus?
Are there any set guidelines on how long a (spoken) corpus stays relevant / is reliable to use? The only major Dutch spoken corpus is from 2004, and I was wondering if using this corpus would cause a ...
0
votes
1
answer
76
views
Where can I get the complete list of word entries present in Merriam Webster's dictionary?
I can't find any list, github repo, etc to find the total list of entries defined in merriam webster. Any ideas?
- 109
11
votes
2
answers
674
views
How to best clean a large historical corpus ridden with OCR errors
Overview:
I have a very large corpus of historical news papers (17th-20th cent.). The word count is about 20 bln. It's raw OCR-ed data in txt-files of about 150 GB. One newspaper issue per file (some ...
- 211
1
vote
0
answers
41
views
Corpus of Taboo card game?
Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
- 576
3
votes
1
answer
113
views
How order of the syntax tree is formally/strictly proven?
An example from https://en.wikipedia.org/wiki/Concrete_syntax_tree : tree of "John hit the ball." .
The tree branches are joined in this order:
{John {hit {the ball}}}.
Ie, "the" ...
- 123
5
votes
1
answer
660
views
Where online can I find a list of all the Hapax Legomena in the Hebrew Bible?
I am looking for a list online of all Hapax Legomena in the Hebrew Bible in text format that I can copy and paste into MS Word for further studies.
I need something that's free and is not in a picture ...
1
vote
2
answers
83
views
Is it possible to test statistical significance of difference between two distance/similarity scores?
I have a corpus consisting of tweets by men and another corpus of female tweets. I was thinking of using a word embeddings approach (e.g., glove and fasttext) and examine the cosine similarity between ...
- 121
2
votes
0
answers
55
views
Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?
If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language?
If yes, do point ...
- 319
1
vote
1
answer
88
views
Praat - window size larger than vowel
I'm doing an acoustic-phonetic analysis of read-speech in an American English corpus.
If we want to set a window/frame size (25 - 30 ms) to cover the majority of vowels in the corpus, but some vowels (...
- 11
2
votes
0
answers
56
views
Word alignment for Switchboard Dialogue Acts Corpus
I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here.
I can see how it is possible to match up every utterance ...
- 576
2
votes
0
answers
184
views
Is the source list of "Oxford English Corpus" public available?
The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
- 121
1
vote
2
answers
173
views
How can I search a corpus for easy sentences?
I would like to search a large corpus for example sentences which contain exclusively or a high proportion of words from the N most common words in some frequency dictionary (the language of the ...
- 179
1
vote
0
answers
109
views
What word category do twink, adonis, muse etc. belong to? [closed]
Terms such as candy, cutie, honey, princess, diamond, queen(?), stud and bunny are terms of endearment (these terms are often used in a relationship to show affection and may also be used, with ...
- 111
1
vote
1
answer
100
views
Spoken Arabic dictionary/corpus?
Is there an existing corpus or dictionary for different spoken Arabic dialects? I'd like to look up a word (in English or MSA or in a spoken dialect) and find where this word or phrase is spoken (and ...
- 173
6
votes
1
answer
77
views
Corpus of English sentences written by non-native speakers?
I am working on a hobby project which tends to help the non-native speakers figure out if the sentences they write are good enough and sounds like written by English native speakers. Any relevant info ...
- 163
0
votes
3
answers
159
views
Which is the most recommended corpus for training a POS Tagger?
The tagger is for English and I will use the universal dependency tag set. I am looking for a corpus that best represents nowadays English.
- 11
1
vote
1
answer
72
views
Comparing corpora
So first, a disclaimer: I'm less than beginner level in everything that has to do with programming.
I'm analysing the occurrences of English code-switching/translanguaging in a particular Polish-...
- 11
2
votes
1
answer
90
views
Searching for an English Whats-app Corpus
I am searching an English Whats-app Corpus in order to analyse a linguistic phenomena. I had some difficulties to find one and maybe some of you can help me out. It is only for corpus driven study and ...
- 72
2
votes
0
answers
47
views
Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?
I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
- 21
6
votes
1
answer
280
views
How to make a Corpus? (selecting a good structure)
I'm making a Corpus.
I have searched for good practices on how to build it.
I've downloaded some Corpus to check their internal structure and the extension file, but I didn't find many.
So, I am lost ...
- 63
1
vote
1
answer
52
views
Suggestion for software to analyze data from social media- please advise
I am searching urgent for a software that I can use to analyse corpora. In short, I would like to have a look at collocations, ellipsis, frequencies, concordances etc. I also want to search for themes ...
- 11
6
votes
1
answer
127
views
What are the conventions for corpus usage?
Suppose I'm working with a language that doesn't have much data available; the largest corpus around is a translation of the Bible, which is free to read online, but is presumably still copyrighted by ...
- 58.8k
1
vote
1
answer
55
views
Is there a comprehensive list of corpora of all the languages (maybe even dialects) of the world?
Is there a comprehensive list of corpora of all the languages of the world?
I've been searching on wikipedia eg. https://en.wikipedia.org/wiki/Czech_National_Corpus. If I scroll to the bottom, I do ...
- 319
2
votes
1
answer
183
views
Does anyone have a link to the American Local News Corpus V1.0?
I found a paper http://www.cs.jhu.edu/~anni/papers/alnc_lrec14.pdf that has made this epic text corpus of over 1 billion words available somewhere, but I can't find it anywhere online.
The paper says ...
- 191
0
votes
1
answer
39
views
Need an online freely available Anaphorically Annotated Corpus of English Language for Identification of Discourse Units
I'm in need of a freely available corpus of English language (which may contain text of any genre) but has to be annotated anaphoriacally i-e. Anaphora resolution should have been performed already.
...
4
votes
0
answers
166
views
Is there an english news corpus available to download for between 1900 and 201X (free or low cost)
I'm attempting a word embedding analysis (think underlying meaning and implications, but computational) of certain keywords through time in the English language, but I am having some difficulty ...
- 191
4
votes
2
answers
462
views
How to interpret this form of Heaps' Law?
Heaps' Law basically is an empirical function that says the number of distinct words you'll find in a document grows as a function to the length of the document. The equation given in the Wikipedia ...
- 143
1
vote
1
answer
28
views
Is there a way to search for language data by context?
For example, if I wanted to see real-life transcriptions of dialogue in situations like a date, a business meeting, two friends walking in a park, etc, just to be able to generate lists of particular ...
- 121
3
votes
1
answer
54
views
Are there any corpora of informal and unstructured text labelled for Named Entity Recognition?
I have been searching since last week for annotated informal texts (with a lot of misspelled words, slang, etc.) to test some Named Entity Recognition tools for research purposes. For example, it ...
- 133
0
votes
2
answers
48
views
Are there corpora which tag phrases by discourse- or conversational-function?
More generally, imagine a corpus which could identify the conversational function of a sentence, such as, "Have a nice day!", in narrow categories like, "ending a conversation", or broad ones like "...
- 121
2
votes
3
answers
443
views
Extensive English word corpora
I am looking for an extensive list of english words[including American and British... just an exhaustive list]. This list of english words should contain an exhaustive collection of all forms of all ...
2
votes
2
answers
134
views
register variation: unbalanced corpus sample
I'm doing a small pilot study on evidentiality in a language I work on.
I'm looking at two evidential markers, -mi that is said to mark the source of knowledge as personal, and -shi that marks ...
- 152