Questions tagged [corpora]

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

57 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6 votes
0 answers
76 views

Is there a threshold of usage of a word, below which it will become an archaism?

University student, beginning to learn about corpora, developing a research question for an assignment. There may well be linguistic concepts and terminology that already cover this idea and that I am ...
user avatar
  • 181
4 votes
0 answers
92 views

Is there an english news corpus available to download for between 1900 and 201X (free or low cost)

I'm attempting a word embedding analysis (think underlying meaning and implications, but computational) of certain keywords through time in the English language, but I am having some difficulty ...
user avatar
  • 191
4 votes
0 answers
57 views

PoS tagger - technical corpus

I try to find corpus with technical language to train my model, but I cannot find it. Do you know if there are any tagged PoS corpus in English related with technical language?
user avatar
  • 41
4 votes
0 answers
578 views

Parallel English-Spanish corpora?

I would like to know of parallel corpora for English-Spanish (that is, texts or language originally in English translated into Spanish and aligned by excerpts of texts) other than those on the Opus ...
user avatar
4 votes
0 answers
559 views

Is there any corpus for idioms?

I'm looking for a corpus for English (American, GB, Australian) idioms. Preferably created manually, because I already have two, but they are rather small and were built semi-automatically.
user avatar
  • 141
4 votes
0 answers
517 views

Tool for manually POS tagging texts

I'm interested if there is a text or set of texts where each word is correctly POS tagged. I know there are algorithms that can associate POS tags to the words, but there are always many of ...
user avatar
  • 261
4 votes
0 answers
359 views

How to decrease CRF++ feature function set?

I have a problem with the CRF++ Package. CRF++ cannot handle large training parts-of-speech corpora (large tagset and large number of words). In fact, the number of feature functions automatically ...
user avatar
  • 141
3 votes
0 answers
52 views

Searching for an English Whats-app Corpus

I am searching an English Whats-app Corpus in order to analyse a linguistic phenomena. I had some difficulties to find one and maybe some of you can help me out. It is only for corpus driven study and ...
user avatar
  • 74
3 votes
0 answers
67 views

Is there a frequency-ranked corpus of Punjabi NAV lemmata?

I am looking for a list of lexical roots for Punjabi, with frequency information whereby one can get an idea what the 1000 most-frequent words of the language are. I need to exclude case forms etc. ...
user avatar
  • 67.7k
3 votes
0 answers
166 views

Distribution of the set of meanings of a given word, in a corpus

I am looking to know about the distribution of word senses, of a particular word in a corpus. How often a word sense will occur out of all the occurrences of that word with any meaning. For example: ...
user avatar
3 votes
0 answers
63 views

Are there any publicly available spell checking corpora?

I need some corpora that contain sentences with misspelled word(s) in order to evaluate the performance of my own spell checking approach. So, the corpora should define the right word alongside the ...
user avatar
  • 139
3 votes
0 answers
208 views

Where can I find Japanese-English (manually) word-aligned corpora?

I'm looking for as many very reliable Japanese-English corpora as possible so I'd like to ask: Are there any other manually aligned corpora besides KFFT's and Utiyama's? What are the most accurate ...
user avatar
3 votes
0 answers
384 views

What is the formula for Usage Rate?

I read about a concept called the "usage rate" (proposed by Juliand and Chang-Rodriguez). It's a method for calculating the frequency of a word in a corpus. What exactly is the "usage rate" and what ...
user avatar
3 votes
1 answer
127 views

Slot filling corpora

I was wondering if anyone knows of slot filling corpora such as ATIS. I tried finding ATIS but since I am not a member of LDC I couldnt get access to it. Do you know of any place where I can find a ...
user avatar
2 votes
0 answers
43 views

Quantitative methodology for contrastive pragmatics in corpus-based settings

I am interested in literature regarding methodology that could be relevant for quantitative research into differences in pragmatic meaning between two 'equivalent' concepts in two languages (in other ...
user avatar
2 votes
0 answers
50 views

Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?

If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language? If yes, do point ...
user avatar
2 votes
0 answers
49 views

Word alignment for Switchboard Dialogue Acts Corpus

I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here. I can see how it is possible to match up every utterance ...
user avatar
  • 576
2 votes
0 answers
128 views

Is the source list of "Oxford English Corpus" public available?

The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
user avatar
  • 121
2 votes
0 answers
392 views

Collocation: interpreting contingency table for log-likelihood measure

I have a conundrum here. I'm attempting to recreate the log-likelihood test example in 'The Cambridge Handbook of Corpus Linguistics (2015). However when I run the same test on the same data I get a ...
user avatar
  • 153
2 votes
0 answers
25 views

How to get data for ACADEMIC + SPOKEN (Academic speech) simultaneously using BNC?

I'm looking for filler words used in academia. Since fillers are mainly used in spoken discourse, I need to look for academic spoken 'texts'. Anyone knows how to do this using BNC? (Explanation: If ...
user avatar
  • 21
2 votes
0 answers
213 views

Differences in collective nouns and agreement between Ame and BrE

I am trying to use two corpora,the BNC and COCA to ascertain the difference in instances of single and plural verb agreement between American and British English. I have some interesting ideas for ...
user avatar
2 votes
0 answers
66 views

What are the subsets of the Eijiro corpus about?

I'm working with 2 japanese corpora called 'waei' and 'reiji'. They are legacy data, so I have no README or else to tell me the topic of either of them. But I need to know what they are about. My ...
user avatar
  • 121
2 votes
0 answers
142 views

Semi-automatic annotation tool like '@nnotate'

Is there an annotation tool for linguistic structure that compares to @nnotate? One of the major features of annotate is the annotation is semi-automatic, i.e., it uses a language model to make ...
user avatar
  • 121
2 votes
0 answers
57 views

Generalisations which a bi-gram probabilistic model might infer from a dataset

I have the following exam question for a machine translation course: From my understanding, I assume the answer is looking for incorrect English grammar which get discovered by bi-grams. So the ...
user avatar
1 vote
0 answers
52 views

What is a good multi language translation system for individual parts of speech such as verbs, nouns, adjectives?

I am developing a language learning tool in Python that generates dual-language books intended to be read as audio books. The system should work by giving single word translations after every ...
user avatar
  • 11
1 vote
0 answers
41 views

Corpus of Taboo card game?

Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
user avatar
  • 576
1 vote
0 answers
46 views

Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?

I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
user avatar
1 vote
0 answers
72 views

How to convert CHAT to TextGrid in a batch?

Siarad corpus is a corpus of spoken Welsh (~40 hours, all transcribed, glossed and translated!). I want to analyze some conversations using Praat. Praat uses the TextGrid format, but the Siarad corpus ...
user avatar
1 vote
0 answers
146 views

Mandarin Chinese and Antconc

Can Antconc be used to build a corpus for Mandarin, Chinese? Specifically, could characters be recognized, or if one were to upload a pinyin text, could tones be displayed? Is this possible?
user avatar
1 vote
0 answers
34 views

Online data giving numbers and frequencies of nouns in some well-known novels?

Are there any freely available online data giving the numbers and frequencies of nouns used in some well-known novels, whether or not the novels are in the public domain? I mean data such as Dan ...
user avatar
1 vote
0 answers
55 views

Corpora for sentence compression

I am looking for corpora for sentence compression. I am mostly interested in English, and any type of sentence. I am aware of: https://github.com/google-research-datasets/sentence-compression ...
user avatar
1 vote
0 answers
26 views

corpus search in Sketch Engine for particular correlative coordinators

Can someone teach me/show me how to search for either...or, neither...or/nor, whether...or and etc. in Sketch engine (in BAWE corpus particularly)? I have read basically every guide and watched a lot ...
user avatar
  • 21
1 vote
0 answers
146 views

Synonym/hyponyms/hypernym detection data set

I am looking for data sets that I could use to train and benchmark a system that detects synonyms, hyponyms and hypernyms in a text. For example, in the sentence In statistics, logistic ...
user avatar
1 vote
0 answers
21 views

How to (automatically ) detect the steady-part of vowels?

Please I need your help ASAP, I need to segment an audio corpus of 4 hours , and I need a tool or script to detect the steady part of vowels and place the boundaries .
user avatar
  • 11
1 vote
0 answers
49 views

Middle and Old English Corpora Distributable Under Open Source License

I'm working on an open source computational linguistics project, and I need Old and Middle English corpora that I can distribute along with the software. Most of the corpora that I've been able to ...
user avatar
  • 111
1 vote
0 answers
217 views

colloquial English corpus with time expressions

I'm looking for a colloquial English corpus that has time expressions marked in it. I don't need the time expressions to be coded in any special way, just that they be marked so I can find them. All ...
user avatar
1 vote
0 answers
56 views

Corpus which includes smileys or even emojies

for a little project in University my group would like to have a look at how emojies or smileys are used in order to support or express irony, sarcasm or similiar. Currently we are trying to find a ...
user avatar
1 vote
0 answers
95 views

Should I use the Log-Likelihood (G2) test to compare word frequency across corpora?

I need to compare the frequency of occurrence of a word in two corpora. Take for example the word fail which occurs 311 times in a corpus containing language from the sciences (2 million tokens) and ...
user avatar
1 vote
0 answers
68 views

Political Advertising Corpus

Is there a large political advertising corpus available for download? I am working on a project but the Wesleyan and University of Wisconson corpus are off limits to me. I sent a request on official ...
user avatar
1 vote
0 answers
87 views

Does corpus linguistics research confirm/disconfirm Bolinger principle?

I remember learning about Bolinger's principle (basically, when there is a verb complement situation like "I like [verb complement]" or "I want to [verb complement]", gerunds are preferred to express ...
user avatar
  • 11
1 vote
0 answers
47 views

Researching extraposition in a syntactic treebank

I'm writing a paper on extraposition in English (and other right-branching discontinuities). I have found a lot of interesting theory on this but the instructions say that if possible concrete corpus ...
user avatar
  • 11
1 vote
0 answers
66 views

Is there a data set of elementary typical phrases translated in different languages?

To be more explicit, I think about a collection of "abstract" sentences which could be categorized in such a way that they would be easily identified in any language. For example, some typical phrases ...
user avatar
  • 111
0 votes
0 answers
34 views

Minimum collocation frequency for corpora?

I have a corpus which contains newspaper articles about trans men and trans women. I want to carry out collocational analysis on this corpus, focusing on verbs. However, I'm not sure what my minimum ...
user avatar
0 votes
0 answers
30 views

Vocabulary Rate for Spanish

I am searching for a resource in Spanish that contains the Spanish Vocabulary Rate, classifying words for their usage level, similar to this Italian one: Nuovo Vocabolario di Base della lingua ...
user avatar
0 votes
0 answers
32 views

How to determine word frequency based on demographic? (Does a public library of word frequencies exist by demographic)

I'm not sure "sign post" questions are appropriate however that is what this is. I am looking for a tool, a means of answering questions such as "Word X is used more by demographic Y than demographic ...
user avatar
0 votes
0 answers
47 views

Corpus Assisted Discourse Studies

I wanted a corpus of recent English fiction (20 century onwards) which I can use as a reference corpus for my research work. It should consist of at least 5 million words. I do not need it in ...
user avatar
0 votes
0 answers
68 views

Vocabulary Wordlist for the elementary school English

I am curious if there is a written English corpus that offers word frequency lists for the elementary stage vocabulary, i.e. K-5 grades? Something like Dr. Edward Fry's sight words, which is updated ...
user avatar
  • 101
0 votes
0 answers
99 views

Corpus for text classification with annotated support phrases

I am looking for corpora for text classification with annotated support phrases. Support phrases = phrases that explain why a text was given a label. For example, if the text is "The patient works ...
user avatar
0 votes
0 answers
113 views

Library of regular expressions for lexical relations in a corpus

I would like to ask if there is a library of regular expressions for mining patterns reflecting lexical semantic relations from a corpus. These patterns should cover a wide variety of aspects synonymy ...
user avatar
  • 171
0 votes
0 answers
155 views

Corpus of tagged text (English newspapers or any tagged text)

I'm developing a system to extract tags from text (English) and currently I have no dataset to test the system and evaluate, could someone point me to a source (preferably a free one) thanks SO I ...
user avatar