Questions tagged [corpora]

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

Filter by
Sorted by
Tagged with
2 votes
0 answers
49 views

COCA corpus: Is it possible to display a list search result as parts of speech?

I want to know what parts of speech (and in what frequency) follow a word or phrase. If I do a list search, for example "for the purpose of *", the result is a list of the actual words in ...
1 vote
1 answer
64 views

Where to find English POS words?

Given a POS (part of speech), tag, and dependency for words, where can I find words that have the same POS, tag, and dependency? I only need a list of words by (pos,tag) and do not need code for this (...
0 votes
1 answer
45 views

I need help with searching a corpus for a complex sentence

I am trying to formulate a search request that will show me example sentences for sentences that contain the phrase "In the beginning" or "At the beginning" followed by the word &...
  • 11
0 votes
0 answers
26 views

Is there any mathematically studied ontology space?

I am considering “ontology generation”. I have not yet read the specifics of these techniques. Still, the point must surely be to identify some kind of cooccurrences / associations between words. ...
0 votes
0 answers
37 views

Where can I find a free/cheap corpus of the 20,000 most frequently used words in the English language? [duplicate]

There are a bunch of large corpora containing millions or billions of words. My needs are simpler: I only need the 20K most used words. Preferably, these words should be most used in the general ...
0 votes
0 answers
15 views

Corpus linguistics: finding a BCCWJ sentence by ID from Universal Dependencies

I am using an online Universal Dependencies search (http://universal.grew.fr/?corpus=UD_Japanese-BCCWJ@2.10) to look at the prevalence of certain grammatical structures in languages including Japanese....
0 votes
1 answer
23 views

Comparing Text Translations by Conducting Corpus Analysis

Provided one has three pieces of text called - “Text A”, “Text B” and “Text C”. Text A is the original text. Text B and Text C are translations of Text A (Text A, being initially written in another ...
1 vote
1 answer
67 views

Can someone share with me an article that studies the decline of by-phrases in the passive?

Can someone share with me an article that studies the decline of by-phrases in the passive? Preferably a corpus-based study, but this is not that relevant
user avatar
2 votes
0 answers
44 views

Quantitative methodology for contrastive pragmatics in corpus-based settings

I am interested in literature regarding methodology that could be relevant for quantitative research into differences in pragmatic meaning between two 'equivalent' concepts in two languages (in other ...
1 vote
0 answers
55 views

What is a good multi language translation system for individual parts of speech such as verbs, nouns, adjectives?

I am developing a language learning tool in Python that generates dual-language books intended to be read as audio books. The system should work by giving single word translations after every ...
  • 11
0 votes
1 answer
58 views

Are alveolo-palatal consonants more likely to be followed by high vowels, whereas retroflex consonants are more likely followed by low vowels?

It seems to me that high voles like i would more naturally follow alveolo-palatal consonants because the need to "spread the lips" (in the popular description of the latter) seem to more ...
  • 175
5 votes
3 answers
978 views

How can I find the frequency for the Russian word "чем-нибудь"?

I've tried searching for the word "чем-нибудь"on the 20,000 word list I found on Wikipedia. You can link directly to it here. For the sake of thoroughness, I even searched the larger 50,...
  • 313
0 votes
1 answer
209 views

Can someone explain to me the Zipf–Mandelbrot law?

I really can't understand. It's about linguistics and I can't understand anything because there are mathematical formulas in it that I can't understand at all. Can anyone explain this with ...
3 votes
1 answer
58 views

Bilingual corpora work from the late 1980s

Does anyone know if the paper by Warwick and Russell (often cited as below) exists? I have looked on the EURALEX website and the authors are not listed as speakers in the 4th Conference. Maybe the ...
  • 171
1 vote
1 answer
108 views

Has anyone ever ranked the prevalence of phones by number of speakers worldwide?

I'm interested in knowing the most-used and least-used phones worldwide. According to Wikipedia, the IPA charts about 140 pulmonic consonants, 80 non-pulmonic consonants, 30 co-articulated consonants, ...
  • 111
0 votes
1 answer
97 views

What is the frequency of color words in English compared to German?

I am writing a term paper about the role of adjectives and gender marking in efficient communication about English and German respectively. And I was wondering if there is any way/website/tool to ...
1 vote
1 answer
45 views

Large parallel corpus of English and Farsi text

hope you are doing well. I am looking for a large parallel corpus of English and Farsi text, professionally written or edited. I would be more than happy to have your suggestions. Thank you all.
0 votes
1 answer
41 views

Where can I find a corpus of sentence with the resulting emotion in french?

I'm working on a sentiment analysis program and I need to validate it using a corpus of sentence with the linked emotions. I need to perform this task on a french corpus and I can't find any on the ...
  • 101
2 votes
1 answer
82 views

Are there any set guidelines for the lifespan of a (spoken) corpus?

Are there any set guidelines on how long a (spoken) corpus stays relevant / is reliable to use? The only major Dutch spoken corpus is from 2004, and I was wondering if using this corpus would cause a ...
0 votes
1 answer
76 views

Where can I get the complete list of word entries present in Merriam Webster's dictionary?

I can't find any list, github repo, etc to find the total list of entries defined in merriam webster. Any ideas?
  • 109
11 votes
2 answers
674 views

How to best clean a large historical corpus ridden with OCR errors

Overview: I have a very large corpus of historical news papers (17th-20th cent.). The word count is about 20 bln. It's raw OCR-ed data in txt-files of about 150 GB. One newspaper issue per file (some ...
  • 211
1 vote
0 answers
41 views

Corpus of Taboo card game?

Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
  • 576
3 votes
1 answer
113 views

How order of the syntax tree is formally/strictly proven?

An example from https://en.wikipedia.org/wiki/Concrete_syntax_tree : tree of "John hit the ball." . The tree branches are joined in this order: {John {hit {the ball}}}. Ie, "the" ...
  • 123
5 votes
1 answer
660 views

Where online can I find a list of all the Hapax Legomena in the Hebrew Bible?

I am looking for a list online of all Hapax Legomena in the Hebrew Bible in text format that I can copy and paste into MS Word for further studies. I need something that's free and is not in a picture ...
1 vote
2 answers
83 views

Is it possible to test statistical significance of difference between two distance/similarity scores?

I have a corpus consisting of tweets by men and another corpus of female tweets. I was thinking of using a word embeddings approach (e.g., glove and fasttext) and examine the cosine similarity between ...
2 votes
0 answers
55 views

Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?

If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language? If yes, do point ...
1 vote
1 answer
88 views

Praat - window size larger than vowel

I'm doing an acoustic-phonetic analysis of read-speech in an American English corpus. If we want to set a window/frame size (25 - 30 ms) to cover the majority of vowels in the corpus, but some vowels (...
  • 11
2 votes
0 answers
56 views

Word alignment for Switchboard Dialogue Acts Corpus

I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here. I can see how it is possible to match up every utterance ...
  • 576
2 votes
0 answers
184 views

Is the source list of "Oxford English Corpus" public available?

The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
  • 121
1 vote
2 answers
173 views

How can I search a corpus for easy sentences?

I would like to search a large corpus for example sentences which contain exclusively or a high proportion of words from the N most common words in some frequency dictionary (the language of the ...
  • 179
1 vote
0 answers
109 views

What word category do twink, adonis, muse etc. belong to? [closed]

Terms such as candy, cutie, honey, princess, diamond, queen(?), stud and bunny are terms of endearment (these terms are often used in a relationship to show affection and may also be used, with ...
  • 111
1 vote
1 answer
100 views

Spoken Arabic dictionary/corpus?

Is there an existing corpus or dictionary for different spoken Arabic dialects? I'd like to look up a word (in English or MSA or in a spoken dialect) and find where this word or phrase is spoken (and ...
6 votes
1 answer
77 views

Corpus of English sentences written by non-native speakers?

I am working on a hobby project which tends to help the non-native speakers figure out if the sentences they write are good enough and sounds like written by English native speakers. Any relevant info ...
0 votes
3 answers
159 views

Which is the most recommended corpus for training a POS Tagger?

The tagger is for English and I will use the universal dependency tag set. I am looking for a corpus that best represents nowadays English.
1 vote
1 answer
72 views

Comparing corpora

So first, a disclaimer: I'm less than beginner level in everything that has to do with programming. I'm analysing the occurrences of English code-switching/translanguaging in a particular Polish-...
2 votes
1 answer
90 views

Searching for an English Whats-app Corpus

I am searching an English Whats-app Corpus in order to analyse a linguistic phenomena. I had some difficulties to find one and maybe some of you can help me out. It is only for corpus driven study and ...
  • 72
2 votes
0 answers
47 views

Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?

I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
6 votes
1 answer
280 views

How to make a Corpus? (selecting a good structure)

I'm making a Corpus. I have searched for good practices on how to build it. I've downloaded some Corpus to check their internal structure and the extension file, but I didn't find many. So, I am lost ...
  • 63
1 vote
1 answer
52 views

Suggestion for software to analyze data from social media- please advise

I am searching urgent for a software that I can use to analyse corpora. In short, I would like to have a look at collocations, ellipsis, frequencies, concordances etc. I also want to search for themes ...
  • 11
6 votes
1 answer
127 views

What are the conventions for corpus usage?

Suppose I'm working with a language that doesn't have much data available; the largest corpus around is a translation of the Bible, which is free to read online, but is presumably still copyrighted by ...
  • 58.8k
1 vote
1 answer
55 views

Is there a comprehensive list of corpora of all the languages (maybe even dialects) of the world?

Is there a comprehensive list of corpora of all the languages of the world? I've been searching on wikipedia eg. https://en.wikipedia.org/wiki/Czech_National_Corpus. If I scroll to the bottom, I do ...
2 votes
1 answer
183 views

Does anyone have a link to the American Local News Corpus V1.0?

I found a paper http://www.cs.jhu.edu/~anni/papers/alnc_lrec14.pdf that has made this epic text corpus of over 1 billion words available somewhere, but I can't find it anywhere online. The paper says ...
  • 191
0 votes
1 answer
39 views

Need an online freely available Anaphorically Annotated Corpus of English Language for Identification of Discourse Units

I'm in need of a freely available corpus of English language (which may contain text of any genre) but has to be annotated anaphoriacally i-e. Anaphora resolution should have been performed already. ...
4 votes
0 answers
166 views

Is there an english news corpus available to download for between 1900 and 201X (free or low cost)

I'm attempting a word embedding analysis (think underlying meaning and implications, but computational) of certain keywords through time in the English language, but I am having some difficulty ...
  • 191
4 votes
2 answers
462 views

How to interpret this form of Heaps' Law?

Heaps' Law basically is an empirical function that says the number of distinct words you'll find in a document grows as a function to the length of the document. The equation given in the Wikipedia ...
  • 143
1 vote
1 answer
28 views

Is there a way to search for language data by context?

For example, if I wanted to see real-life transcriptions of dialogue in situations like a date, a business meeting, two friends walking in a park, etc, just to be able to generate lists of particular ...
3 votes
1 answer
54 views

Are there any corpora of informal and unstructured text labelled for Named Entity Recognition?

I have been searching since last week for annotated informal texts (with a lot of misspelled words, slang, etc.) to test some Named Entity Recognition tools for research purposes. For example, it ...
  • 133
0 votes
2 answers
48 views

Are there corpora which tag phrases by discourse- or conversational-function?

More generally, imagine a corpus which could identify the conversational function of a sentence, such as, "Have a nice day!", in narrow categories like, "ending a conversation", or broad ones like "...
2 votes
3 answers
443 views

Extensive English word corpora

I am looking for an extensive list of english words[including American and British... just an exhaustive list]. This list of english words should contain an exhaustive collection of all forms of all ...
2 votes
2 answers
134 views

register variation: unbalanced corpus sample

I'm doing a small pilot study on evidentiality in a language I work on. I'm looking at two evidential markers, -mi that is said to mark the source of knowledge as personal, and -shi that marks ...
  • 152

1
2 3 4 5