Words and phrases more likely in everyday speech

Question

I'm processing a large corpus for a given language. I've noticed that for certain sections of the corpus, I can go many tens of thousands of words without the word "me" being mentioned. These are obviously the parts of the corpus with more technical data.

I'm primarily interested in words and phrases used in everyday casual speech and am wondering what words i can search for in order to find those parts of the corpus which are more suitable to my particular application.

Here are some of my suggestions: me, you, he, she, them, our, ours, their, theirs, my, your, mine, yours you're, i, "i am", "you are". So I would expect these words to have a higher frequency of occurrence in those parts of the corpus of most relevance. What other words and phrases would you suggest? Is there a danger that I bias my results in a manner I haven't anticipated?

Answer 1

Could your results be biased by English? There could be languages with lower or zero frequency of personal pronouns due to one or all of the reasons below:

1. Personal pronouns are hidden within verb paradigms. E.g. in Spanish one would hardly say yo tengo (I have) but rather just tengo instead, since the meaning of 1st.p.sg. is conveyed by verb ending.

2. Personal pronouns are subject to change. Consider Japanese, Chinese or Basque, where all or some of the personal pronouns vary according to speaker's/addressee's age, gender, social status, etc. Personal pronouns may even be tabooed.

3. Personal pronouns fit a different paradigm. Consider languages with pronouns changing by tense (Kpelle) or with other categories (e.g. inclusive/exclusive dichotomies known for many languages).

4. There are no personal pronouns (although no such a language has been attested for), or they are within a really minimalistic paradigm (e.g. Lardil has only two personal pronouns).

5. The context/corpus covers a limited scope (e.g. there is a sacred language, an illiterate community, bilingual/multilingual community with social stratification by languages, or, to put it simply, the corpus covers a limited scope of all the social registers of a language).