I need a free English language corpus with at least 15 million words. The corpus should contain one or more plain text files. There should be no tagging, just raw text. The corpus should be free. I would prefer if the corpus contained was for modern English, with a mixture of: tv, radio, film, news, fiction, technical etc., or better still, just plain everyday conversation, but this is not a requirement. I will be processing each sentence in the text with the python programming language. Can anyone direct me to such a resource.
The open part of the American National Corpus (OANC) might fulfill your criteria. It contains almost 15 m. words, it's free, and contains conversations and other genres. The spoken part consists mainly of the telephone based Switchboard corpus. If you want more face to face conversations consider adding the Santa Barbara Corpus of Spoken American English.
The OANC comes in versions with different annotation schemes. If you take the version with the least amount of annotation you should be able to strip off all the annotations with a regex find and replace scheme in Python or Notepad++.
Here is a variety of language corpora with millions of sentences each: http://corpora.informatik.uni-leipzig.de/download.html
protected by Alenanno♦ Apr 17 '14 at 16:17
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?