6

I do not have a linguistic background, but I am working on a research project that involves measuring the complexity of news. One of the metrics that I was interested in is the semantic complexity of a text (newspaper article).

I use an automated content analysis method to analyse thousands of articles, so this measure (or index) should be universally applicable to any English language text.

I don't really now if it's possible to measure semantic complexity in such a way, but any suggestions, references to literature etc. would be greatly appreciated. Thanks.

2
  • 1
    You say you "use an automated content analysis method to analyse thousands of articles", but you don't clarify what it does. I may be able to answer your question if you include more details above. For now, I'll point you to software like Boxer and Illinois SRL.
    – prash
    Aug 6 '15 at 0:58
  • Hi, my script runs through thousands of newspaper articles one by one and gathers basic linguistic measures like sentence length, word length, number of words, number of sentences, Flesch Index, lexical diversity, etc.
    – Peter
    Aug 7 '15 at 7:12
5

There is no uniform measure of semantic complexity that you could apply to any text regardless of genre, topic or speaker/listener/reader context. Even within newspaper articles you will find several genres and topic areas where a single measure is unlikely to suffice.

The term "semantic complexity" is used in the literature it generally refers to individual words and categories - e.g. in the context of aphasia.

Perhaps the best place to start your research would be the concept of readability which deals with text complexity in several dimensions by proxy. Another place to look is lexical density.

However, expect this to be a non-trivial problem if you want to go beyond the most simple of measures.

1

Measuring "semantic complexity" is a very vague notion. It would likely be very difficult to define what that means for every imaginable context it would arise.

One possible strategy could break down "semantic complexity" into several, easier to define terms, and then find some way of quantifying and measuring them. Theoretically, this could lead to a workable algorithm.

0

I suggest you research the concept of "semantic entropy". Check out this paper to start: https://pdfs.semanticscholar.org/82b8/9271245e4f813d2494c1c9f37ddfff991e50.pdf

1
  • Welcome to Linguistics! Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.
    – bytebuster
    Jul 8 '18 at 19:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.