I'm working on a NLP project, where we analyze large text samples (think a novel), and produce some metrics that help us answer interesting questions about the text. One of these metrics is average number of independent clauses per compound sentence.
The way I determine that a sentence is compound, as opposed to simple or complex (including complex/compound), is by first using the Stanford Parser to obtain the syntax tree, and then doing a variation of the algorithm suggested here. My algorithm is a bit simpler because I wanted to see some numbers over a variety of texts sooner, before deciding if they were interesting enough to invest time in a more complicated approach.
What I found, across genre and register, was that the average number of independent clauses per compound sentence tended to be around 2. It struck me as weird, because I was expecting more variation, but I don't know if it's actually weird. I tried to find information online about this, but that search didn't produce anything helpful.
Is there any research on the prevalence of different sentence types over, say, a text corpus; furthermore, is there any research on sentence complexity? I want to understand if my algorithm is simply buggy, or if it's reflecting an actual trend that others have already seen as well.