Questions tagged [corpora]
A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.
207
questions
0
votes
0answers
19 views
Potsdam Sentence Corpus - English comparable corpora
Dear fellow Linguists,
I am searching for an English corpus that has a high degree of comparability with the German Potsdam Sentence Corpus. Naturally I am already reading a lot of literature, but I ...
2
votes
1answer
92 views
register variation: unbalanced corpus sample
I'm doing a small pilot study on evidentiality in a language I work on.
I'm looking at two evidential markers, -mi that is said to mark the source of knowledge as personal, and -shi that marks ...
1
vote
0answers
32 views
Corpus of Taboo card game?
Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
3
votes
1answer
80 views
How order of the syntax tree is formally/strictly proven?
An example from https://en.wikipedia.org/wiki/Concrete_syntax_tree : tree of "John hit the ball." .
The tree branches are joined in this order:
{John {hit {the ball}}}.
Ie, "the" ...
3
votes
5answers
6k views
Sounds that humans make with their mouths
I'm interested in a list of sounds that humans make with their mouths.
I'm not looking for
speech sounds
sounds that take a non-average skill (such as beatboxing)
sounds that cannot be made with the ...
4
votes
2answers
303 views
How to interpret this form of Heaps' Law?
Heaps' Law basically is an empirical function that says the number of distinct words you'll find in a document grows as a function to the length of the document. The equation given in the Wikipedia ...
2
votes
2answers
354 views
Where to find/purchase an offline (downloadable) Arabic corpus
I'm developing an application to teach Quran. I need some textual/lingual analysis based on a common and general corpus.
I know about King Abdulaziz City for Science and Technology (KACST) Arabic ...
5
votes
3answers
590 views
Are there tools to build domain specific corpus? [closed]
I would like to build a domain specific corpus. By this I mean a corpus from a specific knowledge area, for instance Law, Psychiatry, or Klingon language.
I am currently using a Wikipedia corpus, and ...
7
votes
2answers
878 views
Is there any corpus for technical English? (E.g., computers, IT, modern technology)
Alternatively, do any larger corpora have categories that pertain to technology, computer science, etc.? Or collections of English documentation and/or manuals related to modern devices, interfaces, ...
0
votes
0answers
16 views
Determining corpora size and participant count
I am setting up a linguistics experiment, involving subjective judgements of synthetic speech. For the sake of example, let's say I'm testing 5 systems, using a Likert scale of 1-5 as a rating. How ...
7
votes
5answers
2k views
Biggest freely available English corpus?
Any help on finding the biggest freely available English corpus that can be used on research?
So far I have found OANC with 15 M words.
4
votes
1answer
125 views
Where online can I find a list of all the Hapax Legomena in the Hebrew Bible?
I am looking for a list online of all Hapax Legomena in the Hebrew Bible in text format that I can copy and paste into MS Word for further studies.
I need something that's free and is not in a picture ...
1
vote
2answers
59 views
Is it possible to test statistical significance of difference between two distance/similarity scores?
I have a corpus consisting of tweets by men and another corpus of female tweets. I was thinking of using a word embeddings approach (e.g., glove and fasttext) and examine the cosine similarity between ...
0
votes
1answer
127 views
Switchboard queries with NXT (the TEXT function)
The NXT query language includes the query function TEXT, which should return the textual content of the variable it applies to. In theory, Switchboard should support this. The file README.SWBD-QUERIES....
1
vote
2answers
3k views
Most commonly used words across languages?
I'm doing my own (very small) corpus analysis, including a word frequency list and would like to compare the top, say, 30 words with other such lists. I know that there are things like the GSL/NGSL, ...
2
votes
0answers
45 views
Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?
If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language?
If yes, do point ...
7
votes
1answer
678 views
How can I calculate if the difference between two word frequencies in one corpus is significant?
I want to study orthographical variants, for example:
Can firefighter and fire-fighter be considered orthographical variants (i.e. the difference in frequency in a corpus is not statistically ...
1
vote
2answers
77 views
How can I search a corpus for easy sentences?
I would like to search a large corpus for example sentences which contain exclusively or a high proportion of words from the N most common words in some frequency dictionary (the language of the ...
0
votes
2answers
44 views
Are there corpora which tag phrases by discourse- or conversational-function?
More generally, imagine a corpus which could identify the conversational function of a sentence, such as, "Have a nice day!", in narrow categories like, "ending a conversation", or broad ones like "...
0
votes
3answers
59 views
Which is the most recommended corpus for training a POS Tagger?
The tagger is for English and I will use the universal dependency tag set. I am looking for a corpus that best represents nowadays English.
2
votes
1answer
29 views
Praat - window size larger than vowel
I'm doing an acoustic-phonetic analysis of read-speech in an American English corpus.
If we want to set a window/frame size (25 - 30 ms) to cover the majority of vowels in the corpus, but some vowels (...
0
votes
0answers
19 views
Need speech corpora for stop-vowel (CV, CVC and VCV) for acoustic phonetics research
I'm looking for a controlled-setting, open-source speech corpus for CV, VCV, CVC syllables. C = stop consonant, and V = all vowels in American English.
More consonantal contexts (fricatives, stops) ...
2
votes
0answers
41 views
Word alignment for Switchboard Dialogue Acts Corpus
I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here.
I can see how it is possible to match up every utterance ...
2
votes
0answers
107 views
Is the source list of “Oxford English Corpus” public available?
The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
4
votes
3answers
1k views
Word frequency list for agglutinative languages like Swahili?
I'm an intermediate Swahili speaker looking to substantially increase the size of my vocabulary over the next couple weeks. However, I want to optimize my learning curve by focusing on the first 2000 ...
2
votes
0answers
83 views
What word category do twink, adonis, muse etc. belong to? [closed]
Terms such as candy, cutie, honey, princess, diamond, queen(?), stud and bunny are terms of endearment (these terms are often used in a relationship to show affection and may also be used, with ...
1
vote
1answer
52 views
Spoken Arabic dictionary/corpus?
Is there an existing corpus or dictionary for different spoken Arabic dialects? I'd like to look up a word (in English or MSA or in a spoken dialect) and find where this word or phrase is spoken (and ...
6
votes
1answer
52 views
Corpus of English sentences written by non-native speakers?
I am working on a hobby project which tends to help the non-native speakers figure out if the sentences they write are good enough and sounds like written by English native speakers. Any relevant info ...
1
vote
1answer
53 views
Comparing corpora
So first, a disclaimer: I'm less than beginner level in everything that has to do with programming.
I'm analysing the occurrences of English code-switching/translanguaging in a particular Polish-...
2
votes
0answers
39 views
Searching for an English Whats-app Corpus
I am searching an English Whats-app Corpus in order to analyse a linguistic phenomena. I had some difficulties to find one and maybe some of you can help me out. It is only for corpus driven study and ...
1
vote
0answers
39 views
Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?
I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
6
votes
1answer
85 views
How to make a Corpus? (selecting a good structure)
I'm making a Corpus.
I have searched for good practices on how to build it.
I've downloaded some Corpus to check their internal structure and the extension file, but I didn't find many.
So, I am lost ...
1
vote
4answers
203 views
Is it possible to build thesaurus automatically?
Is it possible to build thesaurus by simply analyzing lots of text, or it's obligatory should be handmade?
Also a question: is there free-to-download&use English language thesaurus?
1
vote
1answer
45 views
Suggestion for software to analyze data from social media- please advise
I am searching urgent for a software that I can use to analyse corpora. In short, I would like to have a look at collocations, ellipsis, frequencies, concordances etc. I also want to search for themes ...
3
votes
0answers
446 views
Parallel English-Spanish corpora?
I would like to know of parallel corpora for English-Spanish (that is, texts or language originally in English translated into Spanish and aligned by excerpts of texts) other than those on the Opus ...
5
votes
1answer
110 views
What are the conventions for corpus usage?
Suppose I'm working with a language that doesn't have much data available; the largest corpus around is a translation of the Bible, which is free to read online, but is presumably still copyrighted by ...
4
votes
2answers
284 views
JSON for corpora
Do you know any cases of corpus linguistics tools or corpora, using JSON to "mark" or delimited linguistics units in morphology, syntax,semantics? It seems all marking systems use some kind of XML or ...
1
vote
1answer
48 views
Is there a comprehensive list of corpora of all the languages (maybe even dialects) of the world?
Is there a comprehensive list of corpora of all the languages of the world?
I've been searching on wikipedia eg. https://en.wikipedia.org/wiki/Czech_National_Corpus. If I scroll to the bottom, I do ...
10
votes
3answers
3k views
Where to find frequency list of English words from newspapers, books and magazines?
I'd like to download a frequency list of English words that are most common in newspapers, books and magazines. It should contain at least 50.000 words.
Is it possible to download such a list from ...
2
votes
1answer
111 views
Does anyone have a link to the American Local News Corpus V1.0?
I found a paper http://www.cs.jhu.edu/~anni/papers/alnc_lrec14.pdf that has made this epic text corpus of over 1 billion words available somewhere, but I can't find it anywhere online.
The paper says ...
0
votes
1answer
31 views
Need an online freely available Anaphorically Annotated Corpus of English Language for Identification of Discourse Units
I'm in need of a freely available corpus of English language (which may contain text of any genre) but has to be annotated anaphoriacally i-e. Anaphora resolution should have been performed already.
...
3
votes
0answers
52 views
Is there an english news corpus available to download for between 1900 and 201X (free or low cost)
I'm attempting a word embedding analysis (think underlying meaning and implications, but computational) of certain keywords through time in the English language, but I am having some difficulty ...
1
vote
1answer
26 views
Is there a way to search for language data by context?
For example, if I wanted to see real-life transcriptions of dialogue in situations like a date, a business meeting, two friends walking in a park, etc, just to be able to generate lists of particular ...
3
votes
1answer
293 views
Different discounting methods with SRILM toolikt
I want to do some smoothing on a corpus and compare the results.
for train language model without smoothing I used "-addsmooth 0" option
now I should add two different smoothing and compare them, I ...
3
votes
1answer
48 views
Are there any corpora of informal and unstructured text labelled for Named Entity Recognition?
I have been searching since last week for annotated informal texts (with a lot of misspelled words, slang, etc.) to test some Named Entity Recognition tools for research purposes. For example, it ...
2
votes
3answers
162 views
Extensive English word corpora
I am looking for an extensive list of english words[including American and British... just an exhaustive list]. This list of english words should contain an exhaustive collection of all forms of all ...
2
votes
0answers
222 views
Collocation: interpreting contingency table for log-likelihood measure
I have a conundrum here.
I'm attempting to recreate the log-likelihood test example in 'The Cambridge Handbook of Corpus Linguistics (2015). However when I run the same test on the same data I get a ...
5
votes
0answers
50 views
PoS tagger - technical corpus
I try to find corpus with technical language to train my model, but I cannot find it.
Do you know if there are any tagged PoS corpus in English related with technical language?
29
votes
5answers
1k views
Lists of linguistic resources
In the interest of cultivating a professional, academic community, I posted this question on Meta. One comment was to open a community-wiki question inviting others to contribute to a list of academic ...
3
votes
3answers
2k views
Conversational English corpus for download
I have a project which requires a corpus of conversational English in plain text (although I can perform some processing as needed). Since I am a student, I need to find a corpus that is free and ...