New answers tagged

3

As this is the Linguistics site, I'm guessing it is Zipf's Law you are interested in? If you analyze a corpus of text, and count the number of times each word occurs, you get their frequency. You then line then up on the x-axis of a graph, in order of rank (i.e. the most common word comes at x=1, the second most common word at x=2, and the least common word ...


0

If my other answer weren't so long, I would have just edited it, but I think this thread will be easier to read if I just add another answer, so here it is. The day after posting my first answer in this thread, I discovered that my usual tool for searching English and Spanish collocations was not functioning. This made me go searching for another, and, in ...


0

I would love to find something that could mimic what the Google Ngram does, too. Unfortunately, I have yet to find one. In the meantime, here are a couple of things you can do / resources that can get somewhat close. I am going to do this using just a single word. I know that the beauty of the Google Ngram is that it allows you to search for a single ...


2

A standard tool in the search for corpora is the CLARIN virtual language observatory (VLO). Searching for "twitter" and setting the language facet to English gives Twitter sentiment for 15 European languages as the top result. The corpus is under a free licence.


2

Here are two other datasets: Arruda's dataset (5171 messages) NER dataset (7398 messages) The NER dataset has the named entities replaced by entity types, like in [ORGANIZATION], [DATE], etc. The Arruda dataset is mainly English and surprisingly emoji free. Neither of the datasets provide much detail on how they were created. Google has a search engine ...


Top 50 recent answers are included