Questions tagged [corpora]
A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.
216
questions
4
votes
1answer
2k views
Is there a computational method to syllabify English words?
There are straightforward ways to convert English words to phonemes via a dictionary that contains such information. However, is there a way to automatically convert English text into syllables? I.e., ...
3
votes
1answer
288 views
Dimensional reduction with synonomy and polysemy
I was wondering something for a computer science project I am doing. Documents usually have too many distinct words to process on a computer efficiently. One solution is to map all words to a single ...
4
votes
3answers
1k views
Word frequency list for agglutinative languages like Swahili?
I'm an intermediate Swahili speaker looking to substantially increase the size of my vocabulary over the next couple weeks. However, I want to optimize my learning curve by focusing on the first 2000 ...
11
votes
3answers
3k views
Where to find frequency list of English words from newspapers, books and magazines?
I'd like to download a frequency list of English words that are most common in newspapers, books and magazines. It should contain at least 50.000 words.
Is it possible to download such a list from ...
2
votes
2answers
52 views
Query format for NP without subject in PPCEME
As I, not being very familiar with corpus linguistics, read the manual for searching through PPCEME I entered:
node: NP*
print_indices: true
query: (NP-SBJ *PRO*)
without getting a result.
Could ...
9
votes
1answer
236 views
Is there an automatic way of identifying transitive verbs in Computational Linguistics?
Is there any straightforward way of identifying transitive verbs (or sentences containing transitive constructions) in an BrE English text?
I've looked into semantic shallow parsers, such as Semafor, ...
4
votes
1answer
435 views
Is there a standard corpus against which to benchmark mechanical parsers?
I was wondering if there is a published standard corpus of English sentences which are confounding for mechanical parsers. I am looking over sentences given here, and I have the following (very short) ...
0
votes
2answers
305 views
1TB word (1-gram) corpus
Where can I find a 1TB or larger 1-gram corpus?
I am comparing certain structures for term-frequency calculation, but without a significantly large dataset I won't be able to show the non-theoretical ...
19
votes
3answers
8k views
Does anyone know of text message corpora?
I am looking for a large corpus of text messages. By large, I am hoping to have at least 15,000 text messages in my sample. I am fine with combining several smaller corpora into a larger corpus as I ...
9
votes
3answers
2k views
How to work on annotating AND sentence-aligning parallel texts?
There are plenty of software programs facilitating interlinear transcription (e.g. Toolbox, Fieldworks Language Explorer). There's also a number of tools that allow you to work on aligning parallel ...
4
votes
0answers
356 views
How to decrease CRF++ feature function set?
I have a problem with the CRF++ Package. CRF++ cannot handle large training parts-of-speech corpora (large tagset and large number of words).
In fact, the number of feature functions automatically ...
12
votes
1answer
626 views
What are some resources that I can use to gather Twitter data for an NLP project?
I was going to use the Edinburgh Twitter Corpus that is referenced in this paper: The Edinburgh Twitter Corpus. But apparently Twitter has changed their Terms and the corpus is no longer available. ...
28
votes
5answers
1k views
Lists of linguistic resources
In the interest of cultivating a professional, academic community, I posted this question on Meta. One comment was to open a community-wiki question inviting others to contribute to a list of academic ...
7
votes
3answers
969 views
Are there dictionaries like Collins COBUILD for other languages than English? [closed]
Let me explain what I'm after:
monolingual
all examples are taken from a large corpus of contemporary text (the COBUILD series is now using Bank of English)
the senses for each entry are sorted by ...
20
votes
4answers
2k views
Corpus of baby-talk or motherese
Baby-talk or motherese is the language that parents tend to use when addressing preverbal or just-starting-to-speak children. What are the fundamental features or rules that define this subset of a ...
6
votes
2answers
371 views
Where could I find a corpus that is purely descriptive in nature and limited in scope?
I'm trying to build a cognitive model of how people learn a event representation from a sentence describing the event, based on St. John & McClelland's Sentence Gestalt Model.
However, while ...
