How to assess the language register of speech?

Question

I am looking for ways to automatically assess the language register of a person's speech: formal, informal, ... or through some grading mechanism that would associate a 'score' depending on the person's level of language. Imagine Homer Simpson compared to John Steinbeck.

I've read about readability tests such as Flesch-Kincaid or SMOG but find their approach too simple. Counting words and syllables does not account for idioms, vocabulary, sentence structure,...

Is there something beyond comparing the speech with a list of simple words such as the Dale-Chall 3,000 Word List?

Edit: I would like to analyze the transcripts of the GOP and democrats debate and see if anything stands out in the way the candidates express themselves. I'm looking at different things, topic modeling, simple sentence metrics, ... . And Informality analysis seems to be an interesting path to follow and may help emphasize differences between the candidates. The fact that there is no 'settled' metric is all the more interesting since I'll to design and train the method. The paper from Mosquera and Moreda cited by Dominik is exactly what I'm looking for. Very grateful for all your comments and help

Answer 1

Well, here's an idea. I don't have specific references for you, but Linda Shockey did some work at Ohio State comparing vowel formants (and maybe other acoustic measures) in careful versus casual speech. As you might expect, vowel positions in acoustic space tend to be more peripheral (though Shockey's results were much more specific). So what you might try is assessing the carefulness level by how far vowels are from schwa (other than schwa itself), using the first two vowel formant levels to measure.

Answer 2

Flesch-Kincaid, SMOG etc. are measures of text complexity and not register. They only work on written language of certain length (textbooks, newspaper articles, etc.) Register can be inferred from a single word. However, lower readability will probably correlate quite well with high formality. But not in a way that you can simply translate the scores.

The problem with asking for differentiation of register is that there is no objective definition of the different registers (levels of formality). So you'd have to come up with some exemplars of what the different registers represent. You'll also need some measure of correctness (how do you know your results match the levels). Is the appropriateness to the situation? Socio-economic status of the speaker? All of those in turn are hard to measure.

The efforts of measuring levels of formality or register made by people like Halliday or Biber use proxies (like presence of certain words) and get pretty good results. But their measures may not be your measures. Other proxies for levels of formality is the proportion of deictics like pronouns or determiners which can then be used as input for machine learning.

So to get a reasonable answer, you will need to specify what exactly you're trying to achieve and for what purpose? What are your resources for achieving that? If you're looking for a simple piece of software you can load a sample in and get a result, you'll be disappointed. Have a look at this paper which outlines some of the previous efforts in this area. It shows that if your needs are quite general, you can get decent broad results that can improve subsequent analysis. But if you want to get a single score for an individual text, you will probably not get very reliable results.