Linguistics

Questions tagged [corpora]

Ask Question

A corpus (plural corpora) is a database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

207 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

Potsdam Sentence Corpus - English comparable corpora

Dear fellow Linguists, I am searching for an English corpus that has a high degree of comparability with the German Potsdam Sentence Corpus. Naturally I am already reading a lot of literature, but I ...
2
votes
1answer
92 views

register variation: unbalanced corpus sample

I'm doing a small pilot study on evidentiality in a language I work on. I'm looking at two evidential markers, -mi that is said to mark the source of knowledge as personal, and -shi that marks ...
1
vote
0answers
32 views

Corpus of Taboo card game?

Has anyone constructed a corpus of people playing the card game Taboo, in a purely text-based setting, like a computer chat, rather than face-to-face with audio or video? The closest I've found are ...
3
votes
1answer
80 views

How order of the syntax tree is formally/strictly proven?

An example from https://en.wikipedia.org/wiki/Concrete_syntax_tree : tree of "John hit the ball." . The tree branches are joined in this order: {John {hit {the ball}}}. Ie, "the" ...
3
votes
5answers
6k views

Sounds that humans make with their mouths

I'm interested in a list of sounds that humans make with their mouths. I'm not looking for speech sounds sounds that take a non-average skill (such as beatboxing) sounds that cannot be made with the ...
4
votes
2answers
303 views

How to interpret this form of Heaps' Law?

Heaps' Law basically is an empirical function that says the number of distinct words you'll find in a document grows as a function to the length of the document. The equation given in the Wikipedia ...
2
votes
2answers
354 views

Where to find/purchase an offline (downloadable) Arabic corpus

I'm developing an application to teach Quran. I need some textual/lingual analysis based on a common and general corpus. I know about King Abdulaziz City for Science and Technology (KACST) Arabic ...
5
votes
3answers
590 views

Are there tools to build domain specific corpus? [closed]

I would like to build a domain specific corpus. By this I mean a corpus from a specific knowledge area, for instance Law, Psychiatry, or Klingon language. I am currently using a Wikipedia corpus, and ...
7
votes
2answers
878 views

Is there any corpus for technical English? (E.g., computers, IT, modern technology)

Alternatively, do any larger corpora have categories that pertain to technology, computer science, etc.? Or collections of English documentation and/or manuals related to modern devices, interfaces, ...
0
votes
0answers
16 views

Determining corpora size and participant count

I am setting up a linguistics experiment, involving subjective judgements of synthetic speech. For the sake of example, let's say I'm testing 5 systems, using a Likert scale of 1-5 as a rating. How ...
7
votes
5answers
2k views

Biggest freely available English corpus?

Any help on finding the biggest freely available English corpus that can be used on research? So far I have found OANC with 15 M words.
4
votes
1answer
125 views

Where online can I find a list of all the Hapax Legomena in the Hebrew Bible?

I am looking for a list online of all Hapax Legomena in the Hebrew Bible in text format that I can copy and paste into MS Word for further studies. I need something that's free and is not in a picture ...
1
vote
2answers
59 views

Is it possible to test statistical significance of difference between two distance/similarity scores?

I have a corpus consisting of tweets by men and another corpus of female tweets. I was thinking of using a word embeddings approach (e.g., glove and fasttext) and examine the cosine similarity between ...
0
votes
1answer
127 views

Switchboard queries with NXT (the TEXT function)

The NXT query language includes the query function TEXT, which should return the textual content of the variable it applies to. In theory, Switchboard should support this. The file README.SWBD-QUERIES....
1
vote
2answers
3k views

Most commonly used words across languages?

I'm doing my own (very small) corpus analysis, including a word frequency list and would like to compare the top, say, 30 words with other such lists. I know that there are things like the GSL/NGSL, ...
2
votes
0answers
45 views

Is it possible to infer the meaning of a word of a language based on corpus analysis, without prior knowledge of the language?

If I am totally foreign to a language, are there corpus analysis methodologies and theories that I can employ to figure out the meaning of a word in a corpus based on that language? If yes, do point ...
7
votes
1answer
678 views

How can I calculate if the difference between two word frequencies in one corpus is significant?

I want to study orthographical variants, for example: Can firefighter and fire-fighter be considered orthographical variants (i.e. the difference in frequency in a corpus is not statistically ...
1
vote
2answers
77 views

How can I search a corpus for easy sentences?

I would like to search a large corpus for example sentences which contain exclusively or a high proportion of words from the N most common words in some frequency dictionary (the language of the ...
0
votes
2answers
44 views

Are there corpora which tag phrases by discourse- or conversational-function?

More generally, imagine a corpus which could identify the conversational function of a sentence, such as, "Have a nice day!", in narrow categories like, "ending a conversation", or broad ones like "...
0
votes
3answers
59 views

Which is the most recommended corpus for training a POS Tagger?

The tagger is for English and I will use the universal dependency tag set. I am looking for a corpus that best represents nowadays English.
2
votes
1answer
29 views

Praat - window size larger than vowel

I'm doing an acoustic-phonetic analysis of read-speech in an American English corpus. If we want to set a window/frame size (25 - 30 ms) to cover the majority of vowels in the corpus, but some vowels (...
0
votes
0answers
19 views

Need speech corpora for stop-vowel (CV, CVC and VCV) for acoustic phonetics research

I'm looking for a controlled-setting, open-source speech corpus for CV, VCV, CVC syllables. C = stop consonant, and V = all vowels in American English. More consonantal contexts (fricatives, stops) ...
2
votes
0answers
41 views

Word alignment for Switchboard Dialogue Acts Corpus

I am trying to connect the dialogue acts of the Switchboard Dialogue Acts Corpus with the word alignment timing information available here. I can see how it is possible to match up every utterance ...
2
votes
0answers
107 views

Is the source list of “Oxford English Corpus” public available?

The Oxford English Corpus (https://en.wikipedia.org/wiki/Oxford_English_Corpus) is a text corpus of 21st century English, used by the makers of the Oxford English Dictionary and by Oxford University ...
4
votes
3answers
1k views

Word frequency list for agglutinative languages like Swahili?

I'm an intermediate Swahili speaker looking to substantially increase the size of my vocabulary over the next couple weeks. However, I want to optimize my learning curve by focusing on the first 2000 ...
2
votes
0answers
83 views

What word category do twink, adonis, muse etc. belong to? [closed]

Terms such as candy, cutie, honey, princess, diamond, queen(?), stud and bunny are terms of endearment (these terms are often used in a relationship to show affection and may also be used, with ...
1
vote
1answer
52 views

Spoken Arabic dictionary/corpus?

Is there an existing corpus or dictionary for different spoken Arabic dialects? I'd like to look up a word (in English or MSA or in a spoken dialect) and find where this word or phrase is spoken (and ...
6
votes
1answer
52 views

Corpus of English sentences written by non-native speakers?

I am working on a hobby project which tends to help the non-native speakers figure out if the sentences they write are good enough and sounds like written by English native speakers. Any relevant info ...
1
vote
1answer
53 views

Comparing corpora

So first, a disclaimer: I'm less than beginner level in everything that has to do with programming. I'm analysing the occurrences of English code-switching/translanguaging in a particular Polish-...
2
votes
0answers
39 views

Searching for an English Whats-app Corpus

I am searching an English Whats-app Corpus in order to analyse a linguistic phenomena. I had some difficulties to find one and maybe some of you can help me out. It is only for corpus driven study and ...
1
vote
0answers
39 views

Where are some of the biggest freely available English 1 on 1 conversation corpuses that are in plain text preferably?

I'm looking for a free, preferably plain text, 1 on 1 English casual conversation (such as texting back and forth) corpus that requires little to no preprocessing (finding and replacing characters). ...
6
votes
1answer
85 views

How to make a Corpus? (selecting a good structure)

I'm making a Corpus. I have searched for good practices on how to build it. I've downloaded some Corpus to check their internal structure and the extension file, but I didn't find many. So, I am lost ...
1
vote
4answers
203 views

Is it possible to build thesaurus automatically?

Is it possible to build thesaurus by simply analyzing lots of text, or it's obligatory should be handmade? Also a question: is there free-to-download&use English language thesaurus?
1
vote
1answer
45 views

Suggestion for software to analyze data from social media- please advise

I am searching urgent for a software that I can use to analyse corpora. In short, I would like to have a look at collocations, ellipsis, frequencies, concordances etc. I also want to search for themes ...
3
votes
0answers
446 views

Parallel English-Spanish corpora?

I would like to know of parallel corpora for English-Spanish (that is, texts or language originally in English translated into Spanish and aligned by excerpts of texts) other than those on the Opus ...
5
votes
1answer
110 views

What are the conventions for corpus usage?

Suppose I'm working with a language that doesn't have much data available; the largest corpus around is a translation of the Bible, which is free to read online, but is presumably still copyrighted by ...
4
votes
2answers
284 views

JSON for corpora

Do you know any cases of corpus linguistics tools or corpora, using JSON to "mark" or delimited linguistics units in morphology, syntax,semantics? It seems all marking systems use some kind of XML or ...
1
vote
1answer
48 views

Is there a comprehensive list of corpora of all the languages (maybe even dialects) of the world?

Is there a comprehensive list of corpora of all the languages of the world? I've been searching on wikipedia eg. https://en.wikipedia.org/wiki/Czech_National_Corpus. If I scroll to the bottom, I do ...
10
votes
3answers
3k views

Where to find frequency list of English words from newspapers, books and magazines?

I'd like to download a frequency list of English words that are most common in newspapers, books and magazines. It should contain at least 50.000 words. Is it possible to download such a list from ...
2
votes
1answer
111 views

Does anyone have a link to the American Local News Corpus V1.0?

I found a paper http://www.cs.jhu.edu/~anni/papers/alnc_lrec14.pdf that has made this epic text corpus of over 1 billion words available somewhere, but I can't find it anywhere online. The paper says ...
0
votes
1answer
31 views

Need an online freely available Anaphorically Annotated Corpus of English Language for Identification of Discourse Units

I'm in need of a freely available corpus of English language (which may contain text of any genre) but has to be annotated anaphoriacally i-e. Anaphora resolution should have been performed already. ...
3
votes
0answers
52 views

Is there an english news corpus available to download for between 1900 and 201X (free or low cost)

I'm attempting a word embedding analysis (think underlying meaning and implications, but computational) of certain keywords through time in the English language, but I am having some difficulty ...
1
vote
1answer
26 views

Is there a way to search for language data by context?

For example, if I wanted to see real-life transcriptions of dialogue in situations like a date, a business meeting, two friends walking in a park, etc, just to be able to generate lists of particular ...
3
votes
1answer
293 views

Different discounting methods with SRILM toolikt

I want to do some smoothing on a corpus and compare the results. for train language model without smoothing I used "-addsmooth 0" option now I should add two different smoothing and compare them, I ...
3
votes
1answer
48 views

Are there any corpora of informal and unstructured text labelled for Named Entity Recognition?

I have been searching since last week for annotated informal texts (with a lot of misspelled words, slang, etc.) to test some Named Entity Recognition tools for research purposes. For example, it ...
2
votes
3answers
162 views

Extensive English word corpora

I am looking for an extensive list of english words[including American and British... just an exhaustive list]. This list of english words should contain an exhaustive collection of all forms of all ...
2
votes
0answers
222 views

Collocation: interpreting contingency table for log-likelihood measure

I have a conundrum here. I'm attempting to recreate the log-likelihood test example in 'The Cambridge Handbook of Corpus Linguistics (2015). However when I run the same test on the same data I get a ...
5
votes
0answers
50 views

PoS tagger - technical corpus

I try to find corpus with technical language to train my model, but I cannot find it. Do you know if there are any tagged PoS corpus in English related with technical language?
29
votes
5answers
1k views

Lists of linguistic resources

In the interest of cultivating a professional, academic community, I posted this question on Meta. One comment was to open a community-wiki question inviting others to contribute to a list of academic ...
3
votes
3answers
2k views

Conversational English corpus for download

I have a project which requires a corpus of conversational English in plain text (although I can perform some processing as needed). Since I am a student, I need to find a corpus that is free and ...

1
2 3 4 5