I need a free English language corpus with at least 15 million words. The corpus should contain one or more plain text files. There should be no tagging, just raw text. The corpus should be free. I would prefer if the corpus contained was for modern English, with a mixture of: tv, radio, film, news, fiction, technical etc., or better still, just plain everyday conversation, but this is not a requirement. I will be processing each sentence in the text with the python programming language. Can anyone direct me to such a resource.

Corpora containing more than 15 million words are often not freely available due to copyright issues (such as the British National Corpus and the Corpus of Contemporary American English).

The open part of the American National Corpus (OANC) might fulfill your criteria. It contains almost 15 m. words, it's free, and contains conversations and other genres. The spoken part consists mainly of the telephone based Switchboard corpus. If you want more face to face conversations consider adding the Santa Barbara Corpus of Spoken American English.

The OANC comes in versions with different annotation schemes. If you take the version with the least amount of annotation you should be able to strip off all the annotations with a regex find and replace scheme in Python or Notepad++.

  • There's also MICASE. – jlawler Aug 22 '13 at 16:28
  • 2
    Is MICASE available for download? I could only find the concordancer interface on the website. – robert Aug 22 '13 at 16:37
  • Contact them and ask. – jlawler Aug 22 '13 at 17:22

Here is a variety of language corpora with millions of sentences each: http://corpora.informatik.uni-leipzig.de/download.html

  • Possibly paid as it requires written permission and a license. – giorgio79 Oct 31 '15 at 7:58

protected by Alenanno Apr 17 '14 at 16:17

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.