3

I'm working on an application to teach English to people. In this application I need to classify texts based on many factors:

  1. What is the current knowledge the learner
  2. What is his/her interests
  3. How difficult the text is in general
  4. How difficult the text is for him/her

1 and 2 are done. Yet I'm stuck at level 3. I can't figure out a way to determine the complexity of a given text. I Googled around and found some links. But they're not sufficient and usually are not written in plain English, or not intended for algorithmic needs (mostly suited for humans).

I also tried to create a simple algorithm myself. It works but it's far from accurate. My algorithm uses PoS tags, word frequencies in a reference Corpus, length of sentences, length of paragraphs, and some other factors.

But while sentences like to be, or not to be, or everything becomes, nothing is are classified as simple, they are really hard to understand by a layman.

So, do we have a well-known approach on estimating and quantifying a given text's complexity?

3
  • Have a lot of competent readers read them and rank them for difficulty. There is no algorithmic way to measure a concept as vague as "difficulty". Difficult for whom? SInce there is no genetic adaptation to reading, everybody that does learn to read learns to read in their own ways, with their own degree of skills, and their own shortcomings, using whatever rules they can perceive, formulate, and follow. So different people find different things difficult. Multi-coder rankings with high inter-coder reliability is the answer. But perhaps you need a different question.
    – jlawler
    Oct 11 '15 at 20:39
  • @jlawler, thank you for these thoughtful insights. Maybe we can at least try to achieve a little classification. For example, English graded readers might follow a fuzzy approach to determine text's complexity and make them simpler. Oct 12 '15 at 4:19
  • English graded readers are what got us into this mess in the first place. The "grade levels" are guesses made by the ignorant. You might as well count letters in the sentence and say that's "complexity".
    – jlawler
    Oct 12 '15 at 13:02
3

Yes and no. There are a number of well-known algorithms for determining a text's complexity. There are several generations of these and most of them are tied to educational attainment such as years of education, education level studied or age.

They are very much replicable across populations but they do not do a very good job for determining suitability to an individual and/or very short texts.

Most of the algorithms take proxies for semantic and syntactic complexities in the length of words and sentences. They do not look at phonic complexity - which may have to do with things like consonant clusters, etc. but would require more work. The Wikipedia entry on Readability does a good job of summarizing them.

The first generation of these algorithms peaked in the 1960s and 70s and they have been built into many mainstream tools such as Microsoft Word and little free tools can be found online. Since the late 1990s, there have been some efforts in second generation products - see a review by Renlearn (which is a commercial actor in this field so keep that in mind).

The lesson is that you cannot really just make up an algorithm without testing it against readers - and that adding factors to the measure does not necessarily improve reliability.

Things get even more difficult when you try to personalize the measure. You need to know a lot about the user - with beginner or struggling readers, it's their phonic skills which is something we've tried to achieve in our work on the Phonics Engine but then you need a way to really understand the text at the phonic level.

You also need to take into account the reader's interests because if they love dinosaurs a word like Pterodactyl may be actually easy for them. Ultimately, if you want to help individuals, you don't need a tool to help you categorize the texts but to help them access the texts or to help creators create more accessible texts. The tools for this are still not very good and looking at a single dimensional score for texts is more of a hindrance than a help.

0

A complex constituent embedded in the middle of a larger constituent is difficult to understand. Ross discussed this in his dissertation Infinite Syntax (publication title) and offered some tentative suggestions about what makes a constituent complex. McCawley also covers this in his text The Syntactic Phenomena of English, and gives this example, among others:

Mary said that she was tired to John.

which is only marginally acceptable, since "that she was tired" is complex and is embedded in the middle of the verb phrase "said ... to John".

Even before Ross's treatment, Victor Yngve in a study "A Model and an Hypothesis for Language Structure" proposed that (roughly, as I recall it) a large degree of left branching in a construction made it more difficult to process. Yngve's idea has not been popular; still, it was interesting.

2
  • thank you for posting and while I learned from it, it seems to me that it's more like a comment rather than being an answer. Am I right? Oct 12 '15 at 4:21
  • Yes, or at least it doesn't give you a method to follow. If you wanted to refine your measure of difficulty, the references give you some interesting things to try. You'd have to have a parsing of the text, and it would be a big project.
    – Greg Lee
    Oct 12 '15 at 5:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.