Sentence stress detection

Question

I was looking for APIs for the detection of sentence stress, also known as prosodic stress, based on input audio.

(Ideally, I was hoping for a library able to assess the level of prominence of a certain word compared to others in the same sentence, e.g.: on a 0 to 1 scale.)

As far as I can tell, there is currently no such thing.

While word stress (syllable stress) is fairly easy to detect (e.g.: using a pronouncing dictionary and audio alignment), sentence stress detection seems to still have a long way to go.

I was wondering:

• What are some of the challenges behind it?
• What is the likely evolution of the research on the topic within the next few years?

I hope this is not too broad a question.

---

EDIT: I'm looking for this information to automatically show people which word receives the most stress (and if possible which other words receive some degree of emphasis) in sentences with audio, in the context of teaching ESL.

Answer 1

I also don't know of any ready-made tool that does this. It would be very helpful to know roughly what you were hoping to use this for, since that would dictate exactly what kind of tool you would need to use. Praat of course has the ability to quantify many of the phonetic correlates of stress, such as pitch and volume, so if that's all you need then there you go. You did say you were hoping for something to give you some sort of score for the different words in a sentence and Praat won't be able to give you that alone. Unless somebody knows of something that somebody has already made, it's probable that you'd have to make your own.

Some difficulties that spring to mind are (1) that the phonetic correlates of "stress" vary from language to language, though my understanding is that sentence-level stress is more consistent cross-linguistically than say word stress, which isn't even necessarily a thing in some languages, e.g. tonal languages; and (2) that many suprasegmental phonetic cues change over the length of an utterance. E.g., speakers use more air at the beginning of an utterance because they have more air to use, not having spent as much talking yet. This leads to a gradual fall in pitch over the course of an utterance, meaning that often the "stressed" parts of an utterance will be high-pitch only relative to where the pitch would normally be at that point in the sentence, and lower-pitch in absolute terms to everything that came before it. I usually only work with word-level prosody, but I know that for example Mandarin's so-called "rising" tone (tone 2) is often realized with a flat pitch, while the "flat" pitch (tone 1) often falls at a normal rate for its position in the utterance, and its "falling" pitch falls faster (or simply starts lower). I can't say for sure, but I would assume something very similar happens for volume, and there's probably some effect on speech rate that would affect the duration of the stressed word as well.

Answer 2

Searching online returns quite a few results, some of which are quite tailored to your needs:

Tepperman, J., & Narayanan, S. (2005, March). Automatic syllable stress detection using prosodic features for pronunciation evaluation of language learners. In 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) Proceedings (Vol. 1, pp. I-937). IEEE.

This paper presents a new technique for automatic syllable stress detection that is tailored for language-learning purposes. Our method, which uses basic prosodic features and others related to the fundamental frequency slope and RMS energy range, is at least as accurate as an expert human listener, but requires no human supervision other than a pre-defined dictionary of expected lexical stress patterns for all words in the system’s vocabulary. Optimal feature choices exhibited an 87-89% accuracy compared with human-tagged stress labels, exceeding the inter-human agreement commonly held to be about 80%. (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.130.9566&rep=rep1&type=pdf)

Imoto, K., Tsubota, Y., Raux, A., Kawahara, T., & Dantsuji, M. (2002). Modeling and automatic detection of English sentence stress for computer-assisted English prosody learning system. In Seventh International Conference on Spoken Language Processing. (This seems closest to what you need)

We address sentence-level stress detection of English for Computer-Assisted Language Learning (CALL) by Japanese students. Stress models are set up by considering syllable structure and position of the syllable in a phrase, which will provide diagnostic information for students. We also propose a two-stage recognition method that first detects the presence of stress and then identifies the stress level using different weighted combinations of acoustic features. The modeling is coherent with conventional linguistic observations. The method achieves stress recognition rate of 95.1% for native and 84.1% for Japanese speakers. (http://www.cs.cmu.edu/~antoine/papers/icslp2002b.pdf)

Ananthakrishnan, S., & Narayanan, S. S. (2005, March). An automatic prosody recognizer using a coupled multi-stream acoustic model and a syntactic-prosodic language model. In 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) Proceedings (Vol. 1, pp. I-937). IEEE.

In this paper, we build a prosody recognition system that detects stress and prosodic boundaries at the word and syllable level in American English using a coupled Hidden Markov Model (CHMM) to model multiple, asynchronous acoustic feature streams and a syntactic-prosodic model that captures the relationship between the syntax of the utterance and its prosodic structure. Experiments show that the recognizer achieves about 75% agreement on stress labeling and 88% agreement on boundary labeling at the syllable level. (https://sail.usc.edu/publications/files/ananthakrishnanicassp2005.pdf)