Where can I find a 1TB or larger 1-gram corpus?
I am comparing certain structures for term-frequency calculation, but without a significantly large dataset I won't be able to show the non-theoretical speed gains.
Linguistics Stack Exchange is a question and answer site for professional linguists and others with an interest in linguistic research and theory. It only takes a minute to sign up.Sign up to join this community
Mark Davies' version of the Google Books corpus (has more functionality than the Ngram viewer) has 155 billion words, which is getting close to 1TB.
Are the corpora available from Google Books Ngram Viewer not the kind of thing you are looking for?