Questions tagged [corpora]

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

Filter by
Sorted by
Tagged with
1
vote
0answers
131 views

Mandarin Chinese and Antconc

Can Antconc be used to build a corpus for Mandarin, Chinese? Specifically, could characters be recognized, or if one were to upload a pinyin text, could tones be displayed? Is this possible?
0
votes
0answers
28 views

How to determine word frequency based on demographic? (Does a public library of word frequencies exist by demographic)

I'm not sure "sign post" questions are appropriate however that is what this is. I am looking for a tool, a means of answering questions such as "Word X is used more by demographic Y than demographic ...
3
votes
0answers
64 views

Is there a frequency-ranked corpus of Punjabi NAV lemmata?

I am looking for a list of lexical roots for Punjabi, with frequency information whereby one can get an idea what the 1000 most-frequent words of the language are. I need to exclude case forms etc. ...
2
votes
2answers
145 views

glossary/dictionary corpus for NLP task

I am looking for glossary/dictionary corpus in machine readable form. Basically huge file/db which would contain word and its definition. Here is sample: Authenticity: a feature that characterizes ...
3
votes
1answer
378 views

Phonetically annotated speech corpus

Are there any phonetically annotated corpora of accented English speech? Preferably English spoken by native English speakers with a strong accents, such as speakers from a specific region in the UK ...
2
votes
1answer
65 views

where can I find free corpus of spoken disabled people (in english, italian)

I am working on the research project in NLP domain. Basically, we are looking for a free audio corpus of spoken disabled people to study (for instance, aged people, or people has difficulty in normal ...
4
votes
2answers
93 views

Tools and algorithms for finding non-verbatim quotations

I have a textual corpus consisting of about 500 different texts, written in the time span of several centuries. The authors of these texts belonged to the same culture, so they used to read and cite ...
1
vote
0answers
25 views

Online data giving numbers and frequencies of nouns in some well-known novels?

Are there any freely available online data giving the numbers and frequencies of nouns used in some well-known novels, whether or not the novels are in the public domain? I mean data such as Dan ...
0
votes
0answers
42 views

Corpus Assisted Discourse Studies

I wanted a corpus of recent English fiction (20 century onwards) which I can use as a reference corpus for my research work. It should consist of at least 5 million words. I do not need it in ...
1
vote
0answers
52 views

Corpora for sentence compression

I am looking for corpora for sentence compression. I am mostly interested in English, and any type of sentence. I am aware of: https://github.com/google-research-datasets/sentence-compression ...
1
vote
1answer
31 views

Corpora for extractive summaries

I am looking for corpora of extractive summaries. I am mostly interested in English, and any type of document.
0
votes
1answer
6k views

What is the difference between type and token?

My understanding of this is quite vague. Token I understand to be the total number of words in a given text, but type I am not so sure about. If I have a variety of inflected forms (e.g eat, eats, ...
2
votes
2answers
94 views

Command line tools for querying corpuses

I've built myself a little corpus that I'm using for language-learning. However, I'm reaching the limits of standard command-line tool-chain: grep and friends. For example, I might look for ...
1
vote
1answer
43 views

Software for exploring large set of textual data

I am working with large amounts of text and already have a bunch of programmatic instruments to explore them (Python, R, etc). To say the truth, most of the exploration tasks like bigram extraction, ...
4
votes
0answers
474 views

Parallel English-Spanish corpora?

I would like to know of parallel corpora for English-Spanish (that is, texts or language originally in English translated into Spanish and aligned by excerpts of texts) other than those on the Opus ...
4
votes
3answers
2k views

Where can I find a free text corpus for the Hindi language?

I need a text corpus for my NLP project in Hindi, however all the corpora I found are not free for downloading. Is there any free annotated Hindi text corpus at all? If yes, where can I find it?
0
votes
0answers
61 views

Vocabulary Wordlist for the elementary school English

I am curious if there is a written English corpus that offers word frequency lists for the elementary stage vocabulary, i.e. K-5 grades? Something like Dr. Edward Fry's sight words, which is updated ...
1
vote
1answer
69 views

Is there any free, single-word wordlist with CLAWS7 part-of-speech tags?

I am looking for a wordlist which includes single-word entries and their part-of-speech tags in UCREL CLAWS7 format. The list must be comprised of scientifically collected data, for example words ...
1
vote
0answers
21 views

corpus search in Sketch Engine for particular correlative coordinators

Can someone teach me/show me how to search for either...or, neither...or/nor, whether...or and etc. in Sketch engine (in BAWE corpus particularly)? I have read basically every guide and watched a lot ...
1
vote
0answers
35 views

Error of word frequency [closed]

I have created a database of lyrics with some 10000 songs and did a word frequency analysis to find the most common words. Next, i sorted the corpus according to the publication date and measured the ...
2
votes
1answer
68 views

Are there any computer readable corpora for ASL?

I'm looking to see if there are any free-to-use corpora in ASL (any domain is fine), transcribed so a computer can use it. I am interested in doing some analysis on ASL syntax, morphology etc. But I ...
3
votes
1answer
109 views

Classifying the verbs in a small corpus

I have a relatively small language corpus (ca. 250000 tokens). There are two possible stems for a verb in the language: active and passive. Some of the verbs in the language occur only in passive ...
4
votes
1answer
193 views

Where can I find training data for dialects of Hindi?

I am working on a NLP project that aims at identifying different dialects of Hindi language. My über goal is to generate dialogs in once the program identifies which dialect is given. I tried ...
0
votes
0answers
92 views

Corpus for text classification with annotated support phrases

I am looking for corpora for text classification with annotated support phrases. Support phrases = phrases that explain why a text was given a label. For example, if the text is "The patient works ...
1
vote
2answers
391 views

Dictionary building / corpus exploring software

What software do lexicographers use these days to manage and search corpora? Is there a standard solution used by most people, or does every dictionary commission their own software to suit their own ...
1
vote
0answers
141 views

Synonym/hyponyms/hypernym detection data set

I am looking for data sets that I could use to train and benchmark a system that detects synonyms, hyponyms and hypernyms in a text. For example, in the sentence In statistics, logistic ...
2
votes
1answer
241 views

building a corpus from COCA KWIC

I'd like to look at an adjective + clause (adverbial, to-infinitival, -ing) combination in COCA and I'm happy with the KWIC results I got so far. For my further analysis, it would be useful to have ...
4
votes
2answers
289 views

Is there a corpus of Arabic text that doesn't include quotations of the Quran?

I want to search such a corpus to determine the commonality or in other terms the frequency of use of the word Quran القرآن in comparison to The Generous Quran القرآن الكريم. But the condition is that ...
1
vote
1answer
301 views

Gold standard corpus size?

I have a 25 MB corpus (2.300.000 words) to annotate with a rule based system. What is the ideal size of a gold standard corpus that I should create in order to evaluate the tool? Many thanks for your ...
0
votes
0answers
104 views

Library of regular expressions for lexical relations in a corpus

I would like to ask if there is a library of regular expressions for mining patterns reflecting lexical semantic relations from a corpus. These patterns should cover a wide variety of aspects synonymy ...
5
votes
3answers
602 views

Are there tools to build domain specific corpus? [closed]

I would like to build a domain specific corpus. By this I mean a corpus from a specific knowledge area, for instance Law, Psychiatry, or Klingon language. I am currently using a Wikipedia corpus, and ...
7
votes
1answer
713 views

How can I calculate if the difference between two word frequencies in one corpus is significant?

I want to study orthographical variants, for example: Can firefighter and fire-fighter be considered orthographical variants (i.e. the difference in frequency in a corpus is not statistically ...
1
vote
1answer
37 views

How to deal with “it's” when using Miller Newman Friedman list in a corpus?

I'm using their list for a project that incorporates some techniques of authorship analysis. I don't know what to do with "it's", though. I obviously cannot ignore the apostrophe and count it as "...
1
vote
1answer
366 views

Corpus Linguistics: How do I compare date from two corpora correctly?

I want to count the occurrences of the English “wanna” structure diachronically, in order to create a frequency per century chart. My first step was to use the Corpus of Historical American English (...
1
vote
0answers
18 views

How to (automatically ) detect the steady-part of vowels?

Please I need your help ASAP, I need to segment an audio corpus of 4 hours , and I need a tool or script to detect the steady part of vowels and place the boundaries .
1
vote
1answer
456 views

Searching the word occurance in coca (corpus of contemporary American English) including meaning

I searched for a word inviting in coca and I got about 50 thousand usage examples. That's a lot of data. I'm interested in, if there is some possibility to refine the search, especially to search ...
4
votes
2answers
170 views

When analyzing a set of corpora, are there any standard practices with regard to the classification of gerunds?

In the article, "How Many Words Do You Need to Know in Spanish (or any other foreign language)? And WHICH Words Should You Be Learning?" I came upon the following: “Assume that a language learner ...
5
votes
2answers
336 views

Corpus Linguistics: Is it possible to add a tag for “sentence ending”?

I'm new to Corpus Linguistics and I'm writing a paper about the English and Portuguese "because noun", a type of construction such as "I'm going home because GTA5". However, when I try to search this ...
3
votes
0answers
158 views

Distribution of the set of meanings of a given word, in a corpus

I am looking to know about the distribution of word senses, of a particular word in a corpus. How often a word sense will occur out of all the occurrences of that word with any meaning. For example: ...
2
votes
1answer
296 views

Phonetics - English Pronunciation of Vowels Corpus

I am a physics student so forgive me if I'm not very precise with the language. I am currently taking an English course, and my teacher is specialized in teaching pronunciation. I am interested in ...
3
votes
7answers
7k views

Sounds that humans make with their mouths

I'm interested in a list of sounds that humans make with their mouths. I'm not looking for speech sounds sounds that take a non-average skill (such as beatboxing) sounds that cannot be made with the ...
0
votes
1answer
196 views

Is there a resource for English relational nouns

Relational nouns, like brother, mother, capital, advisor, leader are different from sortal words like cup, person, or woman. They differ in the sense that, one cannot decide if x is a brother unto ...
3
votes
2answers
79 views

Comparative Methodology

Firstly many apologies if this question is not appropriate for this forum; I am a mathematician and a mathematics education researcher. However, I am posting on this forum because I am doing research ...
0
votes
1answer
201 views

How to determine who is the author of a message in the nps_chat corpora?

I try to use nps_chat corpora provided by nltk library. I follow the example given here. In particular, I can reproduce this results: >>> from nltk.corpus import nps_chat >>> ...
2
votes
2answers
367 views

Where to find/purchase an offline (downloadable) Arabic corpus

I'm developing an application to teach Quran. I need some textual/lingual analysis based on a common and general corpus. I know about King Abdulaziz City for Science and Technology (KACST) Arabic ...
0
votes
1answer
73 views

word relatedness, defining character of a word

How can NLP tell the word "stool" is closely related to the word "dirty?" When the word relationship is based on a corpus, most likely the word "stool" is related to some medical terms such as "bloody"...
2
votes
1answer
184 views

Open corpora for modern English

I need a corpus of modern English, with part-of-speech tags, in order to train language models, specifically a part-of-speech tagger. The domain I am doing this in is the Hansard, the transcript of ...
0
votes
3answers
141 views

Who is corpus annotator

I'm a beginner of nlp. I'm wondering who is annotator of corpus usually ? Who does actually do works of annotating ? Researchers?
1
vote
2answers
847 views

French corpus with frequency list of POS tagged words (not lemmas)

I'm trying to find a corpus (even purchase it) of French language that has these characteristics: It should has a frequency list List of words (not just lemmas) which are POS tagged Preferably taken ...
3
votes
1answer
425 views

Is the Philippine corpus known as Palito still available for download online?

I have visited http://ccs.dlsu.edu.ph:8086/Palito/, but it is no longer online. SEAlang has the same content, I think, but it isn't downloadable. Is it still available for download online?