Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [corpora]

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

4
votes
1answer
25 views

Looking for Spanish email or chat corpora

I am looking for some corpora containing emails or chats in Spanish.
1
vote
3answers
73 views

Words with the hightest frequency that are not Collocates of each other (inside a text corpora)

Is there a way to obtain a: List, that has 2 words per row Each 2 words are not collocates (or collocations) of each other (ie: are not found within the same phrase inside the entire text corpus, or ...
1
vote
1answer
25 views

Dataset of subject-verb-object/adverb-verb/adjective-noun pairs

Does there exist a dataset which contains a list of subject-verb-object triplets or adverb-verb/adjective-noun pairs that occur within a certain (large) corpus? If not, what would be the most ...
2
votes
1answer
32 views

How to extract example sentences from linguistics papers

Does anyone know of a good tool to grab all example sentences used in a paper so they could be processed via an NLP pipeline? I have udpipe in mind especially: I'd like to take sentences from one or ...
0
votes
0answers
16 views

Corpora with graph data base like neo4j

Relating with corpus linguistics with graph databases, I've seen some examples of using neo4j for store the words of a single phrase in which it is used a relation of --NEXT--> to link one word with ...
4
votes
1answer
44 views

JSON for corpora

Do you know any cases of corpus linguistics tools or corpora, using JSON to "mark" or delimited linguistics units in morphology, syntax,semantics? It seems all marking systems use some kind of XML or ...
0
votes
0answers
23 views

regression analysis

A fellow student advised me, that to make predictions based on diachronic corpus data, one should use the predict formula in Rstudio and create a regression analysis. Can someone please confirm this? ...
1
vote
2answers
36 views

Corpus for simple or basic english

I'm trying to find a good, extensive corpus that uses "simple" or "basic" English, in the sense that it should be easily understandable to people who are not proficient in the English language. For ...
3
votes
2answers
64 views

Which free resource to find word frequency?

My hypothesis is that math jargon is more aliening than other academic fields, and I want to concrete that by an example of the average frequency of the word in these two definitions of "...
2
votes
1answer
35 views

When corpus data suggests a word is common in a language?

I have two questions. If I were to use diachronic corpora to investigate the frequency of usage  for  words within  a word class of English that is mostly archaic today, how do I determine if the ...
1
vote
1answer
38 views

What statistical test is appropriate in a corpus analysis of pronoun resolution preferences?

I have extracted data from a corpus that collected sentences with an antecedent and a next sentence with a pronoun or a stressed pronoun in subject position. Now, the anaphora can either refer back to ...
5
votes
0answers
58 views

Is there a threshold of usage of a word, below which it will become an archaism?

University student, beginning to learn about corpora, developing a research question for an assignment. There may well be linguistic concepts and terminology that already cover this idea and that I am ...
1
vote
0answers
34 views

What is the state of the art for diacritic restoration?

Diacritic restoration is a term for guessing the original form of a text, after it's been converted to ASCII in a lossy way. In Lingála, for example, writing in ASCII merges two pairs of vowels and ...
1
vote
0answers
35 views

Mandarin Chinese and Antconc

Can Antconc be used to build a corpus for Mandarin, Chinese? Specifically, could characters be recognized, or if one were to upload a pinyin text, could tones be displayed? Is this possible?
0
votes
0answers
22 views

How to determine word frequency based on demographic? (Does a public library of word frequencies exist by demographic)

I'm not sure "sign post" questions are appropriate however that is what this is. I am looking for a tool, a means of answering questions such as "Word X is used more by demographic Y than demographic ...
3
votes
0answers
45 views

Is there a frequency-ranked corpus of Punjabi NAV lemmata?

I am looking for a list of lexical roots for Punjabi, with frequency information whereby one can get an idea what the 1000 most-frequent words of the language are. I need to exclude case forms etc. ...
3
votes
2answers
53 views

glossary/dictionary corpus for NLP task

I am looking for glossary/dictionary corpus in machine readable form. Basically huge file/db which would contain word and its definition. Here is sample: Authenticity: a feature that characterizes ...
3
votes
1answer
91 views

Phonetically annotated speech corpus

Are there any phonetically annotated corpora of accented English speech? Preferably English spoken by native English speakers with a strong accents, such as speakers from a specific region in the UK ...
3
votes
1answer
53 views

where can I find free corpus of spoken disabled people (in english, italian)

I am working on the research project in NLP domain. Basically, we are looking for a free audio corpus of spoken disabled people to study (for instance, aged people, or people has difficulty in normal ...
5
votes
2answers
90 views

Tools and algorithms for finding non-verbatim quotations

I have a textual corpus consisting of about 500 different texts, written in the time span of several centuries. The authors of these texts belonged to the same culture, so they used to read and cite ...
0
votes
0answers
28 views

English text corpus related for cooking

I need a dataset of conversations about food preparation or recipe discussions. I found dialogues about movies but not cooking. Is there a resource for such a thing online?
0
votes
0answers
16 views

NLP task of making/evaluating conclusions

I am looking for labelled corpus for NLP making/evaluation conclusions task (I do not know proper name for it). Here is an example of what I mean. Premise: "It is raining outside" Conclusion: "I ...
1
vote
0answers
25 views

Online data giving numbers and frequencies of nouns in some well-known novels?

Are there any freely available online data giving the numbers and frequencies of nouns used in some well-known novels, whether or not the novels are in the public domain? I mean data such as Dan ...
0
votes
0answers
31 views

Corpus Assisted Discourse Studies

I wanted a corpus of recent English fiction (20 century onwards) which I can use as a reference corpus for my research work. It should consist of at least 5 million words. I do not need it in ...
1
vote
0answers
28 views

Corpora for sentence compression

I am looking for corpora for sentence compression. I am mostly interested in English, and any type of sentence. I am aware of: https://github.com/google-research-datasets/sentence-compression ...
1
vote
1answer
31 views

Corpora for extractive summaries

I am looking for corpora of extractive summaries. I am mostly interested in English, and any type of document.
0
votes
1answer
2k views

What is the difference between type and token?

My understanding of this is quite vague. Token I understand to be the total number of words in a given text, but type I am not so sure about. If I have a variety of inflected forms (e.g eat, eats, ...
2
votes
2answers
47 views

Command line tools for querying corpuses

I've built myself a little corpus that I'm using for language-learning. However, I'm reaching the limits of standard command-line tool-chain: grep and friends. For example, I might look for ...
0
votes
1answer
32 views

Software for exploring large set of textual data

I am working with large amounts of text and already have a bunch of programmatic instruments to explore them (Python, R, etc). To say the truth, most of the exploration tasks like bigram extraction, ...
2
votes
0answers
118 views

Parallel English-Spanish corpora?

I would like to know of parallel corpora for English-Spanish (that is, texts or language originally in English translated into Spanish and aligned by excerpts of texts) other than those on the Opus ...
4
votes
3answers
421 views

Where can I find a free text corpus for the Hindi language?

I need a text corpus for my NLP project in Hindi, however all the corpora I found are not free for downloading. Is there any free annotated Hindi text corpus at all? If yes, where can I find it?
0
votes
0answers
35 views

Vocabulary Wordlist for the elementary school English

I am curious if there is a written English corpus that offers word frequency lists for the elementary stage vocabulary, i.e. K-5 grades? Something like Dr. Edward Fry's sight words, which is updated ...
1
vote
0answers
38 views

Is there any free, single-word wordlist with CLAWS7 part-of-speech tags?

I am looking for a wordlist which includes single-word entries and their part-of-speech tags in UCREL CLAWS7 format. The list must be comprised of scientifically collected data, for example words ...
1
vote
0answers
14 views

corpus search in Sketch Engine for particular correlative coordinators

Can someone teach me/show me how to search for either...or, neither...or/nor, whether...or and etc. in Sketch engine (in BAWE corpus particularly)? I have read basically every guide and watched a lot ...
1
vote
0answers
30 views

Error of word frequency [closed]

I have created a database of lyrics with some 10000 songs and did a word frequency analysis to find the most common words. Next, i sorted the corpus according to the publication date and measured the ...
1
vote
1answer
39 views

Are there any computer readable corpora for ASL?

I'm looking to see if there are any free-to-use corpora in ASL (any domain is fine), transcribed so a computer can use it. I am interested in doing some analysis on ASL syntax, morphology etc. But I ...
4
votes
1answer
73 views

Classifying the verbs in a small corpus

I have a relatively small language corpus (ca. 250000 tokens). There are two possible stems for a verb in the language: active and passive. Some of the verbs in the language occur only in passive ...
5
votes
1answer
143 views

Where can I find training data for dialects of Hindi?

I am working on a NLP project that aims at identifying different dialects of Hindi language. My über goal is to generate dialogs in once the program identifies which dialect is given. I tried ...
0
votes
0answers
80 views

Corpus for text classification with annotated support phrases

I am looking for corpora for text classification with annotated support phrases. Support phrases = phrases that explain why a text was given a label. For example, if the text is "The patient works ...
1
vote
2answers
198 views

Dictionary building / corpus exploring software

What software do lexicographers use these days to manage and search corpora? Is there a standard solution used by most people, or does every dictionary commission their own software to suit their own ...
1
vote
0answers
93 views

Synonym/hyponyms/hypernym detection data set

I am looking for data sets that I could use to train and benchmark a system that detects synonyms, hyponyms and hypernyms in a text. For example, in the sentence In statistics, logistic ...
2
votes
1answer
125 views

building a corpus from COCA KWIC

I'd like to look at an adjective + clause (adverbial, to-infinitival, -ing) combination in COCA and I'm happy with the KWIC results I got so far. For my further analysis, it would be useful to have ...
5
votes
2answers
219 views

Is there a corpus of Arabic text that doesn't include quotations of the Quran?

I want to search such a corpus to determine the commonality or in other terms the frequency of use of the word Quran القرآن in comparison to The Generous Quran القرآن الكريم. But the condition is that ...
1
vote
1answer
161 views

Gold standard corpus size?

I have a 25 MB corpus (2.300.000 words) to annotate with a rule based system. What is the ideal size of a gold standard corpus that I should create in order to evaluate the tool? Many thanks for your ...
0
votes
0answers
93 views

Library of regular expressions for lexical relations in a corpus

I would like to ask if there is a library of regular expressions for mining patterns reflecting lexical semantic relations from a corpus. These patterns should cover a wide variety of aspects synonymy ...
5
votes
3answers
257 views

How can I automatically build a domain specific corpus from scratch? [closed]

I would like to know if it is possible to build a domain specific corpus from scratch. By domain specific I mean a corpus which covers a single topic, e.g. European union, basketball, Hollywood, IT, ...
7
votes
1answer
312 views

How can I calculate if the difference between two word frequencies in one corpus is significant?

I want to study orthographical variants, for example: Can firefighter and fire-fighter be considered orthographical variants (i.e. the difference in frequency in a corpus is not statistically ...
1
vote
1answer
36 views

How to deal with “it's” when using Miller Newman Friedman list in a corpus?

I'm using their list for a project that incorporates some techniques of authorship analysis. I don't know what to do with "it's", though. I obviously cannot ignore the apostrophe and count it as "...
3
votes
1answer
154 views

Corpus Linguistics: How do I compare date from two corpora correctly?

I want to count the occurrences of the English “wanna” structure diachronically, in order to create a frequency per century chart. My first step was to use the Corpus of Historical American English (...
1
vote
0answers
16 views

How to (automatically ) detect the steady-part of vowels?

Please I need your help ASAP, I need to segment an audio corpus of 4 hours , and I need a tool or script to detect the steady part of vowels and place the boundaries .