Is there a comprehensive list about which phonemes in the English language can be distinguished via lip-reading and which can't?
-
I am no expert on lip reading, but if there is such a resource, it's not just going to be a flat list. [m] and [b] can both be distinguished from [a] but not from each other. And it's going to be speaker-dependent, since the same phone may be articulated by different speakers in different ways. Even for a specific speaker, "readability" is surely dependent on many linguistic factors, such as speech rate, degree of enunciation, etc., and many non-linguistic factors, such as how close a view of the mouth the lip reader has, what kind of lighting there is, etc.– musicallinguistCommented Jul 18, 2014 at 4:06
-
Oh, and if the resource knows what it's talking about, it's going to list phones, not phonemes. I know someone who pronounces /l/ as [l] in certain environments but as [ɫ] in others, and I reckon the former is easier to distinguish from other phones than the latter, since the front of her tongue is active in the articulation of the former but completely uninvolved in the articulation of the latter.– musicallinguistCommented Jul 18, 2014 at 4:06
-
The phones that you can see, which are generally the labial ones and the rounded ones. Interdental fricatives might qualify, too. Open versus close vowel height can probly be determined from jaw placement, but it's unlikely front, central or mid vowels can be distinguished visually. And syllables will be hard to distinguish and may easily be missed, especially if reduced. Not to mention that different people talk differently, and look different while talking.– jlawlerCommented Jul 18, 2014 at 18:53
Add a comment
|
1 Answer
Have a look into visemes. These are like phonemes, but relate to what the face is doing (esp. the mouth) for each phoneme. More than one phoneme can map to a viseme (e.g. voiced and voiceless consonants have the same viseme as the distinction is due to the vibration of the vocal chords, not the shape of the mouth).
As far as I can tell, although there was earlier research into confusion of similar phonemes, the visemes were first created by early Disney animators. They defined 12/13 facial shapes for the 45 or so English phonemes. They are also used in text-to-speech applications.
Other classifications can have a different number of visemes,, for example the Microsoft Speech Application Programming Interface (SAPI) defines 21 visemes.
There does not appear to be any clear consensus on the phoneme to viseme classification. I'm not sure what it is like in the lip reading field (a Gamasutra article mentioned there were 18 lip reading visemes, but I was not able to find a reference to this in my brief search, although visemes were mentioned in connection to lip reading in other results).
There is the problem of context, where the shape of the mouth is influenced by the previous and next phoneme.
In addition to this, the simplified viseme model does not account for the shape of the cheeks and jaws, nor the position of the tongue, which can all be visual clues.
The ability to lip read will take the movement of the facial features into account and make inferences based on context (e.g. a t/d shape after [shi:] could be sheet or she'd). This is similar to how speech recognition software works, but that operates on the acoustic (audio) signal.
