A database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

learn more… | top users | synonyms (1)

1
vote
2answers
16 views

French corpus with frequency list of POS tagged words (not lemmas)

I'm trying to find a corpus (even purchase it) of French language that has these characteristics: It should has a frequency list List of words (not just lemmas) which are POS tagged Preferably taken ...
1
vote
0answers
16 views

Is the Philippine corpus known as Palito still available for download online?

I have visited http://ccs.dlsu.edu.ph:8086/Palito/, but it is no longer online. SEAlang has the same content, I think, but it isn't downloadable. Is it still available for download online?
1
vote
1answer
20 views

List of English words with usage samples

For my learning tool project I need a free English word list with the usage sample of each word and/or its definition. For example: /word/ Apple /phrase/ This apples are big and red. /definition/ ...
3
votes
0answers
30 views

Are there any publicly available spell checking corpora?

I need some corpora that contain sentences with misspelled word(s) in order to evaluate the performance of my own spell checking approach. So, the corpora should define the right word alongside the ...
2
votes
2answers
56 views

I am building a chatbot and I need corpora

I am building a chatbot and I need corpora to train them. I need corpora that is used as general chatting. I would be happy if anyone would help me with it.
4
votes
1answer
66 views

Parts-of-speech tagging and finding relevant phrases in documents

I've got a corpus of half a million text documents. I'd like to identify phrases in each document that are the most descriptive with which to build tag clouds. Let's say that I identify the most ...
2
votes
0answers
13 views

How to get data for ACADEMIC + SPOKEN (Academic speech) simultaneously using BNC?

I'm looking for filler words used in academia. Since fillers are mainly used in spoken discourse, I need to look for academic spoken 'texts'. Anyone knows how to do this using BNC? (Explanation: If ...
2
votes
0answers
41 views

Differences in collective nouns and agreement between Ame and BrE

I am trying to use two corpora,the BNC and COCA to ascertain the difference in instances of single and plural verb agreement between American and British English. I have some interesting ideas for ...
1
vote
2answers
34 views

Criminal Text Corpora

My goal is to bolster a database with charge type categories and decipher between, criminals, victims, interested parties, and suspects. I have searched for a corpora of text that may already have ...
1
vote
1answer
28 views

Corpus for anaphora resolution [closed]

I need a test and train corpus for my project containing anaphora. It should atleast have 100 sentences. please help asap?
1
vote
0answers
30 views

Middle and Old English Corpora Distributable Under Open Source License

I'm working on an open source computational linguistics project, and I need Old and Middle English corpora that I can distribute along with the software. Most of the corpora that I've been able to ...
0
votes
0answers
32 views

Corpus of tagged text (English newspapers or any tagged text)

I'm developing a system to extract tags from text (English) and currently I have no dataset to test the system and evaluate, could someone point me to a source (preferably a free one) thanks SO I ...
0
votes
1answer
14 views

Related to Corpa Query software

Some time ago, a colleague at an institution I used to work at gave a presentation to the faculty on using corpora queries. She used [I think a website, though it could have been a software] where you ...
0
votes
0answers
31 views

how may I identify informal synonyms and abbreviations in a corpus?

I am interested in atherogenesis which is the initiating process for atherosclerosis which in turn, can lead to ischaemic/ischemic stroke. Is there a recognised process/algorithm - ideally using a ...
0
votes
0answers
25 views

Significance Testing in corpus linguistics for corpus comparison

I would like to compare two corpora, one consisting of Wikipedia articles and one made up of articles in printed encyclopedias. The corpus is thematically comparable because I always chose the same ...
1
vote
0answers
35 views

colloquial English corpus with time expressions

I'm looking for a colloquial English corpus that has time expressions marked in it. I don't need the time expressions to be coded in any special way, just that they be marked so I can find them. All ...
1
vote
1answer
26 views

Open corpora of personal names

The only open corpus of personal names of which I am aware is the Names Corpus by Mark Kantrowitz and Bill Ross (which can also be found in the Python Natural Language Toolkit (NLTK)). It is much ...
1
vote
0answers
24 views

Corpus with sentences translated to English

Does anyone know of a corpus I could use in my research?, for any language, where each sentence is translated correctly into English. In particular, the corpus should contain primarily everyday spoken ...
1
vote
0answers
23 views

Corpus which includes smileys or even emojies

for a little project in University my group would like to have a look at how emojies or smileys are used in order to support or express irony, sarcasm or similiar. Currently we are trying to find a ...
1
vote
2answers
55 views

Corpora Tools for Cataloguing and Phonetic Markup?

I'm creating an annotated corpus of poetry annotated with accent, alliteration, rhythm, and phonetics. I'd like it to handle and input poetry in multiple languages but still be able to add markup (...
2
votes
2answers
138 views

Looking for korean text corpus

I'm looking for a list of text corpus for korean. Is there any web site where I can find it? Regards.
3
votes
2answers
46 views

American English Corpus with spoken language exchange

I'm looking for an American English Corpus with spoken language exchange. There are many terms used. E.g. eTandem, tandem, etc. TIA
1
vote
0answers
26 views

Should I use the Log-Likelihood (G2) test to compare word frequency across corpora?

I need to compare the frequency of occurrence of a word in two corpora. Take for example the word fail which occurs 311 times in a corpus containing language from the sciences (2 million tokens) and ...
1
vote
0answers
32 views

Why doesn't GeneTag use BIO for its annotations?

GeneTag (1) is a tagged corpus for gene/protein named entity recognition. The corpus is annotated as follows: P00089778A0000 The_TAG concentration_TAG of_TAG alpha_GENE1 2_GENE1 -_GENE1 ...
1
vote
0answers
24 views

Political Advertising Corpus

Is there a large political advertising corpus available for download? I am working on a project but the Wesleyan and University of Wisconson corpus are off limits to me. I sent a request on official ...
0
votes
2answers
92 views

Do we have a taxonomy other than part of speech?

I'm trying to create an educational system, and I would like to have more taxonomies than a simple part of speech. For example, I would like to be able to categorize words into: Tree => object ...
2
votes
2answers
44 views

What are some canonical or seminal corpus linguistic studies using the Brown Corpus?

I'm looking for some good works of corpus linguistics using the Brown Corpus. Does anyone know of an important early work, and/or an important recent work that uses it?
2
votes
0answers
31 views

Slot filling corpora

I was wondering if anyone knows of slot filling corpora such as ATIS. I tried finding ATIS but since I am not a member of LDC I couldnt get access to it. Do you know of any place where I can find a ...
0
votes
0answers
24 views

Phrase suggestion model using a gold standard corpus

There are already spell checking models available which help us to find the suggested correct spellings based on a corpus of trained correct spellings. Can the granularity be increased to "word" from ...
1
vote
2answers
316 views

Most commonly used words across languages?

I'm doing my own (very small) corpus analysis, including a word frequency list and would like to compare the top, say, 30 words with other such lists. I know that there are things like the GSL/NGSL, ...
2
votes
2answers
218 views

Do standard corpus analysis tools like AntConc and Wordsmith work for foreign languages?

Can I use tools like AntConc and Wordsmith and produce decent results when analyzing languages like French, German or Spanish? Would it deal with special symbols like "à", "ü", "ß", "ç" and similar ...
2
votes
1answer
68 views

Question type corpus

I'm looking for a corpus that contains questions and question types. Like in this article, but they didn't attach any link to their corpus. http://polibits.gelbukh.com/2009_40/40_01.pdf
1
vote
0answers
53 views

Does corpus linguistics research confirm/disconfirm Bolinger principle?

I remember learning about Bolinger's principle (basically, when there is a verb complement situation like "I like [verb complement]" or "I want to [verb complement]", gerunds are preferred to express ...
1
vote
1answer
37 views

How do you search for nouns that contain the sequence of letters sb in the Corpus of Contemporary American English?

I am trying to design a word list for a phonetics project. I can use wildcard searches in the corpus such as * sb * which will return all words that contain this sequence of letters (either medially ...
0
votes
1answer
127 views

Special Characters and Uppercase Frequency?

Is there any way to have a good heuristic study on the frequency of special characters and uppercase ? Would studying a freely available corpus (around 15m words) generates a good estimation? Edit: ...
1
vote
2answers
270 views

Biggest freely available English corpus?

Any help on finding the biggest freely available English corpus that can be used on research? So far I have found OANC with 15 M words.
0
votes
1answer
49 views

Reason for using 4- and 5-grams for co-occurrence analysis

I am trying to replicate the results of a paper which didn't share its data methods that clearly. It makes use of the Web 1T 5-gram Version 1 data set collected from Google to expand 72 words into 964 ...
3
votes
0answers
37 views

Different discounting methods with SRILM toolikt

I want to do some smoothing on a corpus and compare the results. for train language model without smoothing I used "-addsmooth 0" option now I should add two different smoothing and compare them, I ...
2
votes
2answers
175 views

Conversational English corpus for download

I have a project which requires a corpus of conversational English in plain text (although I can perform some processing as needed). Since I am a student, I need to find a corpus that is free and ...
1
vote
0answers
33 views

Researching extraposition in a syntactic treebank

I'm writing a paper on extraposition in English (and other right-branching discontinuities). I have found a lot of interesting theory on this but the instructions say that if possible concrete corpus ...
1
vote
1answer
106 views

Evaluate idea to autobuild russian-english parallel corpus

I was looking for russian-english parallel corpus, but have found none publicly available to the moment. If you know one, or parallel text database, I would be very grateful for one. (I heard of ...
1
vote
3answers
130 views

Is it possible to build thesaurus automatically?

Is it possible to build thesaurus by simply analyzing lots of text, or it's obligatory should be handmade? Also a question: is there free-to-download&use English language thesaurus?
0
votes
2answers
51 views

Expected Frequency

I am reading an article where the authors mention something they call the expected frequency of a word in a corpus without explaining or giving a definition of what this means. The article in ...
4
votes
0answers
84 views

Is there any corpus for technical English? (E.g., computers, IT, modern technology)

Alternatively, do any larger corpora have categories that pertain to technology, computer science, etc.? Or collections of English documentation and/or manuals related to modern devices, interfaces, ...
2
votes
1answer
101 views

Word difficulty as a function of frequency

What is known in general about the relationship between word difficulty and word frequency? That is, for a given word, suppose I know the frequency with which it (or its word stem) is encountered in ...
5
votes
1answer
79 views

How should grammatical errors be treated in parsing and tokenisation?

I'm writing a tokenizer/parser thing to parse & diagram a bunch of sentences gathered from ordinary language use. Most of it parses just fine by following rules, but from time to time there are ...
0
votes
1answer
57 views

German sentence corpus with cloze norms

I wish to conduct research on German based on cloze probabilities. Cloze probabilities are not trivial to collect, so I am asking if there exist any publicly available corpora of German sentences with ...
6
votes
1answer
268 views

copyright issues when making a corpus from the web

I realize that this is probably technically off-topic as it relates to copyright legalities of corpus construction. However, it is a practically very important issue for corpus linguistics and so I ...
2
votes
1answer
129 views

Time annotated corpus: plain text english corpus

I am working on how entities take a new sense over time. I am trying to find out a large english corpus (free to download) which should have the time annotation of the origin of the text. I suppose ...
0
votes
0answers
71 views

How to annotate a corpus of written discourse for pragmatic analysis (speech acts)?

I am currently creating a small-ish corpus of online conversations that I wish to analyze quantitatively and qualitatively from a pragmatic perspective (speech acts level mainly). The problem is that ...