6

Any help on finding the biggest freely available English corpus that can be used on research?

So far I have found OANC with 15 M words.

  • Does it matter what language it's in? – user6726 May 10 '15 at 21:26
  • 1
    Take a look at the Global Web-Based English Corpus. 1.9 billion words. – robert May 10 '15 at 22:42
  • Yes English, we want to use it for letter frequency studies. The online ones usually don't let you search on letters – Tt22 May 11 '15 at 4:03
  • Are you just trying to estimate letter frequencies? If so, I think a much smaller corpus will be fine. – Russell Richie Feb 25 '16 at 5:59
  • @robert While that corpus is searchable, the content is not freely available. Each corpus at that site costs hundreds of dollars. – Mark Jan 27 '18 at 0:15
4

Sketch Engine, a corpus manager and text analysis software, provides a few corpora with open access for research on https://app.sketchengine.eu/#open The largest English corpus (freely available) is ACL Anthology Reference Corpus with 62 million words.

On the other hand, you can try 30-day free trial of Sketch Engine and search one of the biggest English corpus which currently exists with over 35 billion words, see more at https://www.sketchengine.eu/timestamped-english-corpus/

| improve this answer | |
3

Can't beat the Global Web-Based English Corpus proposed by robert---but here are is another big one:

A Wikipedia dump is also huge ...

| improve this answer | |
2

Westbury labs provides a ~1 billion word Wikipedia dump of all articles with greater than 1000 words from 2010: http://www.psych.ualberta.ca/~westburylab/downloads/westburylab.wikicorp.download.html

BYU has a larger dump (1.9 bn) from 2014, but it's not available for download.

| improve this answer | |
1

The COBUILD corpus (18M tokens) is available through WebCelex, if the arcane user interface isn't a deal-breaker. It's valuable more for its extensive manual annotations than its size, with quite a lot of morphological and phonological information available.

(It's smaller than most of the others listed here, but seems worth mentioning, since it's larger than the OANC mentioned in the question and is well-annotated.)

| improve this answer | |
0

I found the Exquisite Corpus and it's freely avalaible. A detail of the sources can be seen here. I really don't know the exact size, but it's on the billions' scale.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.