18

I am looking for a large corpus of text messages. By large, I am hoping to have at least 15,000 text messages in my sample. I am fine with combining several smaller corpora into a larger corpus as I will also be adding thousands of text messages relating to the patterns of interest in my research.

Clarification on Requirements:

  • Text messages should predominantly be in English (US/American ideally), although a mixture of Spanish (Mexican Spanish) is also good.
  • The corpus can be either free or available for a reasonable fee.
  • They should be text messages, as I am specifically looking for euphemisms and slang used in text messaging as well as emoticons.
3
  • 1
    It isn't exactly what you're asking for but have you considered using the Twitter API to build a corpus of tweets?
    – acattle
    Oct 28 '12 at 11:00
  • not free, but look at this: catalog.ldc.upenn.edu/LDC2017T07
    – Daniel
    Oct 26 '17 at 4:23
  • @Daniel this is in Egyptian Arabic, not English (see first bulleted requirement) in question
    – Dan
    Nov 24 '17 at 22:53
11

I only found this corpus on the NUS (National University of Singapore) site, but luckily it has a lot of entries.

It has a download1 for a corpus containing ~10,000 text messages, which was the original corpus.

But if you go to this page2, there is a table listing the corpora (they range from ~10,000 to ~51,000 text messages) available for download. The top line being the most recent corpus there; you can download either in XML or SQL, or download the statistics.

I must remind you (and everyone who uses it) that you should make sure to follow the instructions asked by the researchers, in case you choose it.

Update: please see the comment below my answer.


Notes: It looks like the download site is down, instead you can download all the data from their Github Repo

6
  • 7
    @Dan O'Day I'm one of organizers of the NUS SMS Corpus, referenced in this answer. We had a recent paper in Language Resources and Evaluation that describes all of the SMS corpora we found, paid or free, that are of some non-trivial scale (you can get the preprint from the corpus website, linked in the answer). Unfortunately, none of them really match your U.S. Eng/Spa (Mexican) requirements. The NUS corpus' English messages are largely from Singaporean university students, so the language you'll see used are quite a bit different than those you would see in U.S. SMS. Oct 27 '12 at 14:32
  • @majnemɪzdæn Consider accepting the answer if it solves your question. :)
    – Alenanno
    Sep 30 '14 at 10:29
  • @Alenanno my apologies, I vacillate only because technically this did not meet my requirements (the Singaporean English didn't work for my requirements). The reality is that there likely is not a corpus that does. I ended up beginning to create one, but do not have enough samples yet to make it all that significant. I did upvote this answer, however - but perhaps another corpus will come to light?
    – Dan
    Sep 30 '14 at 13:06
  • @majnemɪzdæn Don't worry, there is nothing forcing you to accept, especially if you're not 100% satisfied, but I doubt there's a corpus that will, as you said.
    – Alenanno
    Sep 30 '14 at 14:49
  • 1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.