I'm working on an application to teach English to people. In this application I need to classify texts based on many factors:
- What is the current knowledge the learner
- What is his/her interests
- How difficult the text is in general
- How difficult the text is for him/her
1 and 2 are done. Yet I'm stuck at level 3. I can't figure out a way to determine the complexity of a given text. I Googled around and found some links. But they're not sufficient and usually are not written in plain English, or not intended for algorithmic needs (mostly suited for humans).
I also tried to create a simple algorithm myself. It works but it's far from accurate. My algorithm uses PoS tags, word frequencies in a reference Corpus, length of sentences, length of paragraphs, and some other factors.
But while sentences like to be, or not to be, or everything becomes, nothing is are classified as simple, they are really hard to understand by a layman.
So, do we have a well-known approach on estimating and quantifying a given text's complexity?