I'm doing a small pilot study on evidentiality in a language I work on.
I'm looking at two evidential markers, -mi that is said to mark the source of knowledge as personal, and -shi that marks knowledge as coming from another source (someone else).
I'm exploring how to test these assumptions quantitatively. Specifically by register.
I have some narrative texts, some are narratives are personal experiences others are second hand accounts of personal experience. All are from the same speaker. In other words, some texts are the speaker talking about her own experience, others are her retelling other experiences.
The hypothesis I am testing is that her personal experience texts will have more -mi's and less -shi's, and the secondhand stories will have more -shi's and less -mi's.
My corpus is small, but I want to use a sample to do a power analysis to determine how many texts I will eventually need to investigate this more deeply. For now, I have 10 total texts that almost match up in size:
firsthand personal experience narratives: Text 1a: 344 words Text 2a: 568 words Text 3a: 1228 words Text 4a: 2316 words Text 5a: **2389 words** secondhand personal experience narratives: Text 1b: 344 words Text 2b: 599 words Text 3b: 1165 words Text 4b: 1794 words Text 5b: **7522 words**
You can see that 1-3 match up pretty well in terms of word count, 4 shows more disparity, but in 5, 5b is about three times the size of 5a.
I don't have any texts in the firsthand register that come close to that size, 5a is the largest. Also all of the texts in the secondhand category are the only the texts I have for that category, so I can't choose another one.
Should I just omit it all together and use a 4v4 sample? Or compensate by having more texts in the firsthand category to bring the word counts level? I'm hesitant about doing that though. From what I've read in Doug Biber's articles, number of texts is more important than word count, but at the same time apparently word count doesn't matter for certain features? Could I do something like log-transform the data?
I'm not sure what to do here, or if there are additional considerations about the situations I haven't thought of.