Questions tagged [corpora]

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

Filter by
Sorted by
Tagged with
1
vote
1answer
42 views

What statistical test is appropriate in a corpus analysis of pronoun resolution preferences?

I have extracted data from a corpus that collected sentences with an antecedent and a next sentence with a pronoun or a stressed pronoun in subject position. Now, the anaphora can either refer back to ...
1
vote
1answer
31 views

Corpora for extractive summaries

I am looking for corpora of extractive summaries. I am mostly interested in English, and any type of document.
1
vote
1answer
44 views

Software for exploring large set of textual data

I am working with large amounts of text and already have a bunch of programmatic instruments to explore them (Python, R, etc). To say the truth, most of the exploration tasks like bigram extraction, ...
1
vote
1answer
79 views

Is there any free, single-word wordlist with CLAWS7 part-of-speech tags?

I am looking for a wordlist which includes single-word entries and their part-of-speech tags in UCREL CLAWS7 format. The list must be comprised of scientifically collected data, for example words ...
1
vote
1answer
37 views

How to deal with "it's" when using Miller Newman Friedman list in a corpus?

I'm using their list for a project that incorporates some techniques of authorship analysis. I don't know what to do with "it's", though. I obviously cannot ignore the apostrophe and count it as "...
1
vote
1answer
500 views

Searching the word occurance in coca (corpus of contemporary American English) including meaning

I searched for a word inviting in coca and I got about 50 thousand usage examples. That's a lot of data. I'm interested in, if there is some possibility to refine the search, especially to search ...
1
vote
2answers
919 views

French corpus with frequency list of POS tagged words (not lemmas)

I'm trying to find a corpus (even purchase it) of French language that has these characteristics: It should has a frequency list List of words (not just lemmas) which are POS tagged Preferably taken ...
1
vote
2answers
91 views

Corpora Tools for Cataloguing and Phonetic Markup?

I'm creating an annotated corpus of poetry annotated with accent, alliteration, rhythm, and phonetics. I'd like it to handle and input poetry in multiple languages but still be able to add markup (...
1
vote
1answer
295 views

What are lexical and morpho-syntactic alternations?

I ought to write a paper with corpus-based analysis of a lexical or morpho-syntactic alternation. In other words, the paper should deal with two (or more) nearly synonymous lexical items or morpho-...
1
vote
1answer
178 views

How is the dependency of corpus size and observed ngrams?

I would like to know how the relation of corpus size to observed ngrams is(especially unigrams). Since the word frequency follows a Zipf distribution, i would expect some kind of logarithmic relation ...
1
vote
1answer
35 views

Large parallel corpus of English and Farsi text

hope you are doing well. I am looking for a large parallel corpus of English and Farsi text, professionally written or edited. I would be more than happy to have your suggestions. Thank you all.
1
vote
1answer
55 views

Are there any set guidelines for the lifespan of a (spoken) corpus?

Are there any set guidelines on how long a (spoken) corpus stays relevant / is reliable to use? The only major Dutch spoken corpus is from 2004, and I was wondering if using this corpus would cause a ...
1
vote
0answers
36 views

Corpus of Taboo card game?

Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
1
vote
0answers
41 views

Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?

I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
1
vote
0answers
48 views

How to convert CHAT to TextGrid in a batch?

Siarad corpus is a corpus of spoken Welsh (~40 hours, all transcribed, glossed and translated!). I want to analyze some conversations using Praat. Praat uses the TextGrid format, but the Siarad corpus ...
1
vote
0answers
135 views

Mandarin Chinese and Antconc

Can Antconc be used to build a corpus for Mandarin, Chinese? Specifically, could characters be recognized, or if one were to upload a pinyin text, could tones be displayed? Is this possible?
1
vote
0answers
25 views

Online data giving numbers and frequencies of nouns in some well-known novels?

Are there any freely available online data giving the numbers and frequencies of nouns used in some well-known novels, whether or not the novels are in the public domain? I mean data such as Dan ...
1
vote
0answers
53 views

Corpora for sentence compression

I am looking for corpora for sentence compression. I am mostly interested in English, and any type of sentence. I am aware of: https://github.com/google-research-datasets/sentence-compression ...
1
vote
0answers
23 views

corpus search in Sketch Engine for particular correlative coordinators

Can someone teach me/show me how to search for either...or, neither...or/nor, whether...or and etc. in Sketch engine (in BAWE corpus particularly)? I have read basically every guide and watched a lot ...
1
vote
0answers
36 views

Error of word frequency [closed]

I have created a database of lyrics with some 10000 songs and did a word frequency analysis to find the most common words. Next, i sorted the corpus according to the publication date and measured the ...
1
vote
0answers
143 views

Synonym/hyponyms/hypernym detection data set

I am looking for data sets that I could use to train and benchmark a system that detects synonyms, hyponyms and hypernyms in a text. For example, in the sentence In statistics, logistic ...
1
vote
0answers
18 views

How to (automatically ) detect the steady-part of vowels?

Please I need your help ASAP, I need to segment an audio corpus of 4 hours , and I need a tool or script to detect the steady part of vowels and place the boundaries .
1
vote
0answers
41 views

Middle and Old English Corpora Distributable Under Open Source License

I'm working on an open source computational linguistics project, and I need Old and Middle English corpora that I can distribute along with the software. Most of the corpora that I've been able to ...
1
vote
0answers
203 views

colloquial English corpus with time expressions

I'm looking for a colloquial English corpus that has time expressions marked in it. I don't need the time expressions to be coded in any special way, just that they be marked so I can find them. All ...
1
vote
0answers
53 views

Corpus which includes smileys or even emojies

for a little project in University my group would like to have a look at how emojies or smileys are used in order to support or express irony, sarcasm or similiar. Currently we are trying to find a ...
1
vote
0answers
92 views

Should I use the Log-Likelihood (G2) test to compare word frequency across corpora?

I need to compare the frequency of occurrence of a word in two corpora. Take for example the word fail which occurs 311 times in a corpus containing language from the sciences (2 million tokens) and ...
1
vote
0answers
65 views

Political Advertising Corpus

Is there a large political advertising corpus available for download? I am working on a project but the Wesleyan and University of Wisconson corpus are off limits to me. I sent a request on official ...
1
vote
0answers
82 views

Does corpus linguistics research confirm/disconfirm Bolinger principle?

I remember learning about Bolinger's principle (basically, when there is a verb complement situation like "I like [verb complement]" or "I want to [verb complement]", gerunds are preferred to express ...
1
vote
0answers
45 views

Researching extraposition in a syntactic treebank

I'm writing a paper on extraposition in English (and other right-branching discontinuities). I have found a lot of interesting theory on this but the instructions say that if possible concrete corpus ...
1
vote
0answers
62 views

Is there a data set of elementary typical phrases translated in different languages?

To be more explicit, I think about a collection of "abstract" sentences which could be categorized in such a way that they would be easily identified in any language. For example, some typical phrases ...
0
votes
1answer
133 views

Can someone explain to me the Zipf–Mandelbrot law?

I really can't understand. It's about linguistics and I can't understand anything because there are mathematical formulas in it that I can't understand at all. Can anyone explain this with ...
0
votes
3answers
145 views

Who is corpus annotator

I'm a beginner of nlp. I'm wondering who is annotator of corpus usually ? Who does actually do works of annotating ? Researchers?
0
votes
2answers
305 views

1TB word (1-gram) corpus

Where can I find a 1TB or larger 1-gram corpus? I am comparing certain structures for term-frequency calculation, but without a significantly large dataset I won't be able to show the non-theoretical ...
0
votes
3answers
92 views

Which is the most recommended corpus for training a POS Tagger?

The tagger is for English and I will use the universal dependency tag set. I am looking for a corpus that best represents nowadays English.
0
votes
2answers
44 views

Are there corpora which tag phrases by discourse- or conversational-function?

More generally, imagine a corpus which could identify the conversational function of a sentence, such as, "Have a nice day!", in narrow categories like, "ending a conversation", or broad ones like "...
0
votes
1answer
7k views

What is the difference between type and token?

My understanding of this is quite vague. Token I understand to be the total number of words in a given text, but type I am not so sure about. If I have a variety of inflected forms (e.g eat, eats, ...
0
votes
1answer
720 views

Special Characters and Uppercase Frequency?

Is there any way to have a good heuristic study on the frequency of special characters and uppercase ? Would studying a freely available corpus (around 15m words) generates a good estimation? Edit: ...
0
votes
1answer
130 views

Switchboard queries with NXT (the TEXT function)

The NXT query language includes the query function TEXT, which should return the textual content of the variable it applies to. In theory, Switchboard should support this. The file README.SWBD-QUERIES....
0
votes
1answer
72 views

What is the frequency of color words in English compared to German?

I am writing a term paper about the role of adjectives and gender marking in efficient communication about English and German respectively. And I was wondering if there is any way/website/tool to ...
0
votes
1answer
39 views

Where can I get the complete list of word entries present in Merriam Webster's dictionary?

I can't find any list, github repo, etc to find the total list of entries defined in merriam webster. Any ideas?
0
votes
1answer
32 views

Need an online freely available Anaphorically Annotated Corpus of English Language for Identification of Discourse Units

I'm in need of a freely available corpus of English language (which may contain text of any genre) but has to be annotated anaphoriacally i-e. Anaphora resolution should have been performed already. ...
0
votes
1answer
209 views

Is there a resource for English relational nouns

Relational nouns, like brother, mother, capital, advisor, leader are different from sortal words like cup, person, or woman. They differ in the sense that, one cannot decide if x is a brother unto ...
0
votes
1answer
211 views

How to determine who is the author of a message in the nps_chat corpora?

I try to use nps_chat corpora provided by nltk library. I follow the example given here. In particular, I can reproduce this results: >>> from nltk.corpus import nps_chat >>> ...
0
votes
1answer
73 views

word relatedness, defining character of a word

How can NLP tell the word "stool" is closely related to the word "dirty?" When the word relationship is based on a corpus, most likely the word "stool" is related to some medical terms such as "bloody"...
0
votes
1answer
24 views

Related to Corpa Query software

Some time ago, a colleague at an institution I used to work at gave a presentation to the faculty on using corpora queries. She used [I think a website, though it could have been a software] where you ...
0
votes
2answers
350 views

Do we have a taxonomy other than part of speech?

I'm trying to create an educational system, and I would like to have more taxonomies than a simple part of speech. For example, I would like to be able to categorize words into: Tree => object ...
0
votes
2answers
137 views

Expected Frequency

I am reading an article where the authors mention something they call the expected frequency of a word in a corpus without explaining or giving a definition of what this means. The article in ...
0
votes
1answer
198 views

German sentence corpus with cloze norms

I wish to conduct research on German based on cloze probabilities. Cloze probabilities are not trivial to collect, so I am asking if there exist any publicly available corpora of German sentences with ...
0
votes
1answer
30 views

Where can I find a corpus of sentence with the resulting emotion in french?

I'm working on a sentiment analysis program and I need to validate it using a corpus of sentence with the linked emotions. I need to perform this task on a french corpus and I can't find any on the ...
0
votes
0answers
56 views

Potsdam Sentence Corpus - English comparable corpora

Dear fellow Linguists, I am searching for an English corpus that has a high degree of comparability with the German Potsdam Sentence Corpus. Naturally I am already reading a lot of literature, but I ...