Questions tagged [corpora]

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

56 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
0answers
71 views

Is there a threshold of usage of a word, below which it will become an archaism?

University student, beginning to learn about corpora, developing a research question for an assignment. There may well be linguistic concepts and terminology that already cover this idea and that I am ...
4
votes
0answers
54 views

PoS tagger - technical corpus

I try to find corpus with technical language to train my model, but I cannot find it. Do you know if there are any tagged PoS corpus in English related with technical language?
4
votes
0answers
514 views

Parallel English-Spanish corpora?

I would like to know of parallel corpora for English-Spanish (that is, texts or language originally in English translated into Spanish and aligned by excerpts of texts) other than those on the Opus ...
4
votes
0answers
551 views

Is there any corpus for idioms?

I'm looking for a corpus for English (American, GB, Australian) idioms. Preferably created manually, because I already have two, but they are rather small and were built semi-automatically.
4
votes
0answers
505 views

Tool for manually POS tagging texts

I'm interested if there is a text or set of texts where each word is correctly POS tagged. I know there are algorithms that can associate POS tags to the words, but there are always many of ...
4
votes
0answers
356 views

How to decrease CRF++ feature function set?

I have a problem with the CRF++ Package. CRF++ cannot handle large training parts-of-speech corpora (large tagset and large number of words). In fact, the number of feature functions automatically ...
3
votes
0answers
45 views

Searching for an English Whats-app Corpus

I am searching an English Whats-app Corpus in order to analyse a linguistic phenomena. I had some difficulties to find one and maybe some of you can help me out. It is only for corpus driven study and ...
3
votes
0answers
61 views

Is there an english news corpus available to download for between 1900 and 201X (free or low cost)

I'm attempting a word embedding analysis (think underlying meaning and implications, but computational) of certain keywords through time in the English language, but I am having some difficulty ...
3
votes
0answers
64 views

Is there a frequency-ranked corpus of Punjabi NAV lemmata?

I am looking for a list of lexical roots for Punjabi, with frequency information whereby one can get an idea what the 1000 most-frequent words of the language are. I need to exclude case forms etc. ...
3
votes
0answers
161 views

Distribution of the set of meanings of a given word, in a corpus

I am looking to know about the distribution of word senses, of a particular word in a corpus. How often a word sense will occur out of all the occurrences of that word with any meaning. For example: ...
3
votes
0answers
55 views

Are there any publicly available spell checking corpora?

I need some corpora that contain sentences with misspelled word(s) in order to evaluate the performance of my own spell checking approach. So, the corpora should define the right word alongside the ...
3
votes
0answers
204 views

Where can I find Japanese-English (manually) word-aligned corpora?

I'm looking for as many very reliable Japanese-English corpora as possible so I'd like to ask: Are there any other manually aligned corpora besides KFFT's and Utiyama's? What are the most accurate ...
3
votes
0answers
376 views

What is the formula for Usage Rate?

I read about a concept called the "usage rate" (proposed by Juliand and Chang-Rodriguez). It's a method for calculating the frequency of a word in a corpus. What exactly is the "usage rate" and what ...
3
votes
1answer
121 views

Slot filling corpora

I was wondering if anyone knows of slot filling corpora such as ATIS. I tried finding ATIS but since I am not a member of LDC I couldnt get access to it. Do you know of any place where I can find a ...
2
votes
0answers
46 views

Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?

If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language? If yes, do point ...
2
votes
0answers
44 views

Word alignment for Switchboard Dialogue Acts Corpus

I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here. I can see how it is possible to match up every utterance ...
2
votes
0answers
109 views

Is the source list of "Oxford English Corpus" public available?

The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
2
votes
0answers
302 views

Collocation: interpreting contingency table for log-likelihood measure

I have a conundrum here. I'm attempting to recreate the log-likelihood test example in 'The Cambridge Handbook of Corpus Linguistics (2015). However when I run the same test on the same data I get a ...
2
votes
0answers
21 views

How to get data for ACADEMIC + SPOKEN (Academic speech) simultaneously using BNC?

I'm looking for filler words used in academia. Since fillers are mainly used in spoken discourse, I need to look for academic spoken 'texts'. Anyone knows how to do this using BNC? (Explanation: If ...
2
votes
0answers
200 views

Differences in collective nouns and agreement between Ame and BrE

I am trying to use two corpora,the BNC and COCA to ascertain the difference in instances of single and plural verb agreement between American and British English. I have some interesting ideas for ...
2
votes
0answers
64 views

What are the subsets of the Eijiro corpus about?

I'm working with 2 japanese corpora called 'waei' and 'reiji'. They are legacy data, so I have no README or else to tell me the topic of either of them. But I need to know what they are about. My ...
2
votes
0answers
380 views

I am looking for an Arabic ngram corpus

I am working in a project where i need to use an ngram model. So, i want to know if an Arabic ngram corpus exist. I have tried to find a corpus but all my researches failed. I know that for languages ...
2
votes
0answers
134 views

Semi-automatic annotation tool like '@nnotate'

Is there an annotation tool for linguistic structure that compares to @nnotate? One of the major features of annotate is the annotation is semi-automatic, i.e., it uses a language model to make ...
2
votes
0answers
54 views

Generalisations which a bi-gram probabilistic model might infer from a dataset

I have the following exam question for a machine translation course: From my understanding, I assume the answer is looking for incorrect English grammar which get discovered by bi-grams. So the ...
1
vote
1answer
55 views

Are there any set guidelines for the lifespan of a (spoken) corpus?

Are there any set guidelines on how long a (spoken) corpus stays relevant / is reliable to use? The only major Dutch spoken corpus is from 2004, and I was wondering if using this corpus would cause a ...
1
vote
0answers
36 views

Corpus of Taboo card game?

Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
1
vote
0answers
41 views

Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?

I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
1
vote
0answers
47 views

How to convert CHAT to TextGrid in a batch?

Siarad corpus is a corpus of spoken Welsh (~40 hours, all transcribed, glossed and translated!). I want to analyze some conversations using Praat. Praat uses the TextGrid format, but the Siarad corpus ...
1
vote
0answers
135 views

Mandarin Chinese and Antconc

Can Antconc be used to build a corpus for Mandarin, Chinese? Specifically, could characters be recognized, or if one were to upload a pinyin text, could tones be displayed? Is this possible?
1
vote
0answers
25 views

Online data giving numbers and frequencies of nouns in some well-known novels?

Are there any freely available online data giving the numbers and frequencies of nouns used in some well-known novels, whether or not the novels are in the public domain? I mean data such as Dan ...
1
vote
0answers
53 views

Corpora for sentence compression

I am looking for corpora for sentence compression. I am mostly interested in English, and any type of sentence. I am aware of: https://github.com/google-research-datasets/sentence-compression ...
1
vote
0answers
23 views

corpus search in Sketch Engine for particular correlative coordinators

Can someone teach me/show me how to search for either...or, neither...or/nor, whether...or and etc. in Sketch engine (in BAWE corpus particularly)? I have read basically every guide and watched a lot ...
1
vote
0answers
143 views

Synonym/hyponyms/hypernym detection data set

I am looking for data sets that I could use to train and benchmark a system that detects synonyms, hyponyms and hypernyms in a text. For example, in the sentence In statistics, logistic ...
1
vote
0answers
18 views

How to (automatically ) detect the steady-part of vowels?

Please I need your help ASAP, I need to segment an audio corpus of 4 hours , and I need a tool or script to detect the steady part of vowels and place the boundaries .
1
vote
0answers
41 views

Middle and Old English Corpora Distributable Under Open Source License

I'm working on an open source computational linguistics project, and I need Old and Middle English corpora that I can distribute along with the software. Most of the corpora that I've been able to ...
1
vote
0answers
200 views

colloquial English corpus with time expressions

I'm looking for a colloquial English corpus that has time expressions marked in it. I don't need the time expressions to be coded in any special way, just that they be marked so I can find them. All ...
1
vote
0answers
53 views

Corpus which includes smileys or even emojies

for a little project in University my group would like to have a look at how emojies or smileys are used in order to support or express irony, sarcasm or similiar. Currently we are trying to find a ...
1
vote
0answers
92 views

Should I use the Log-Likelihood (G2) test to compare word frequency across corpora?

I need to compare the frequency of occurrence of a word in two corpora. Take for example the word fail which occurs 311 times in a corpus containing language from the sciences (2 million tokens) and ...
1
vote
0answers
65 views

Political Advertising Corpus

Is there a large political advertising corpus available for download? I am working on a project but the Wesleyan and University of Wisconson corpus are off limits to me. I sent a request on official ...
1
vote
0answers
82 views

Does corpus linguistics research confirm/disconfirm Bolinger principle?

I remember learning about Bolinger's principle (basically, when there is a verb complement situation like "I like [verb complement]" or "I want to [verb complement]", gerunds are preferred to express ...
1
vote
0answers
45 views

Researching extraposition in a syntactic treebank

I'm writing a paper on extraposition in English (and other right-branching discontinuities). I have found a lot of interesting theory on this but the instructions say that if possible concrete corpus ...
1
vote
0answers
62 views

Is there a data set of elementary typical phrases translated in different languages?

To be more explicit, I think about a collection of "abstract" sentences which could be categorized in such a way that they would be easily identified in any language. For example, some typical phrases ...
0
votes
0answers
54 views

Potsdam Sentence Corpus - English comparable corpora

Dear fellow Linguists, I am searching for an English corpus that has a high degree of comparability with the German Potsdam Sentence Corpus. Naturally I am already reading a lot of literature, but I ...
0
votes
0answers
29 views

How to determine word frequency based on demographic? (Does a public library of word frequencies exist by demographic)

I'm not sure "sign post" questions are appropriate however that is what this is. I am looking for a tool, a means of answering questions such as "Word X is used more by demographic Y than demographic ...
0
votes
0answers
42 views

Corpus Assisted Discourse Studies

I wanted a corpus of recent English fiction (20 century onwards) which I can use as a reference corpus for my research work. It should consist of at least 5 million words. I do not need it in ...
0
votes
0answers
61 views

Vocabulary Wordlist for the elementary school English

I am curious if there is a written English corpus that offers word frequency lists for the elementary stage vocabulary, i.e. K-5 grades? Something like Dr. Edward Fry's sight words, which is updated ...
0
votes
0answers
92 views

Corpus for text classification with annotated support phrases

I am looking for corpora for text classification with annotated support phrases. Support phrases = phrases that explain why a text was given a label. For example, if the text is "The patient works ...
0
votes
0answers
107 views

Library of regular expressions for lexical relations in a corpus

I would like to ask if there is a library of regular expressions for mining patterns reflecting lexical semantic relations from a corpus. These patterns should cover a wide variety of aspects synonymy ...
0
votes
0answers
149 views

Corpus of tagged text (English newspapers or any tagged text)

I'm developing a system to extract tags from text (English) and currently I have no dataset to test the system and evaluate, could someone point me to a source (preferably a free one) thanks SO I ...
0
votes
0answers
111 views

how may I identify informal synonyms and abbreviations in a corpus?

I am interested in atherogenesis which is the initiating process for atherosclerosis which in turn, can lead to ischaemic/ischemic stroke. Is there a recognised process/algorithm - ideally using a ...