I'm not sure "sign post" questions are appropriate however that is what this is. I am looking for a tool, a means of answering questions such as "Word X is used more by demographic Y than demographic Z." to achieve this end I think the following is required: A large diverse collection of documents and sources with meta-data which can then be used as selection criteria to define a corpus.

An example of such selection might be: Include/Exclude social media from x,y,z sources. Include/Exclude academic papers. Include/Exclude magazines (by name), newspapers (by name), blogs. Only select where the authors geographic zone is known or self identified, and restricted to some geographic zone(City, country). Restrict search by gender. Use audio to text capture from media sources? Only include books found under the following sections: a,b,c. etc.

Then apply word frequency counts by methodology (first x words from text), sum boolean word occurrence by document, in the case of audio-to-text indicate audio sample size per count. etc.

Does such a tool exist?

  • It should be noted that I am not particularly interested in any particular demographic, although at this time I would only be interested in English language. – Quaternion Feb 23 '18 at 21:26
  • 1
    What you're talking about is called Sociolinguistics. But it doesn't necessarily pay any attention to officially defined "demographics", since they rarely represent reality, and sociolinguistics is about individual and speech-group variation and change, as measured in actual speech. – jlawler Feb 23 '18 at 21:36
  • 1
    Yes, that sounds like what I'm looking for. Also your definition of demographics is in line with my expectations. – Quaternion Feb 23 '18 at 22:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.