Why is recognition based on detecting individual phonemes, and not chunks such as morphemes or syllables?..

My question were in relation to human perception, and the existence of phonetic boundary, that is visible when one change the VOT. VOT < 30 is recognised as /da/ and above is it detected as /ta/.

enter image description here

So some form of categorical perception is clear at this level, so why not use it in computers?

  • There are too many morphemes or syllables.
    – Greg Lee
    Commented Sep 2, 2017 at 14:01
  • 3
    Do you have a concrete reason to believe that recognition is actually based on phonemes?
    – user6726
    Commented Sep 2, 2017 at 14:44
  • 3
    Incidentally, you should clarify whether you mean "human speech perception", vs. "speech to text programming".
    – user6726
    Commented Sep 2, 2017 at 17:25
  • @user6726 I added a concrete example from the book E. B. Goldstein,Sensation and Perception. 2009, p. 496, In which they explain the existence of categorical perception.. Commented Sep 2, 2017 at 18:52
  • 1
    Are you equating recognition and perception (and where does discrimination fit in?). Most people can recognize that اخذ is not the same as أعطينا, and I'm betting many people here can recognize what language that is, even if they don't know the phonemes that are represented. Is your question specifically about categorial perception in speech?
    – user6726
    Commented Sep 2, 2017 at 21:15

2 Answers 2


Assuming the goal of writing a speech recognition program that does what the human mind does, a large non-linguistic front end must be dealt with first (a front end that is decidedly not part of linguistics), namely, how does the human auditory system work, starting at the ear? What exactly hits the brain coming from the primary auditory cortex, or later? If you can determine what kind of inputs are specifically linguistically processed (not undifferentiated sound which includes music and car horns), then the question of language perception (hence recognition) might be addressable.

There is also a completely non-linguistic question that has to be resolved (jointly by philosophy and experimental psychology), namely, what is "perception" or "recognition"? What do humans perceive (or, what is the nature of a "perception"), and what do they infer from what they perceive? In seeing a dog, do we "perceive" numerous minuscule attributes and infer that there is a dog, or do we perceive a dog, and infer numerous minuscule attributes? These are not linguistic question, but they need to be answered before answering the question w.r.t. language.

The basic linguistic question analogous to the preceding is, do we perceive features and infer segments and so on, or do we perceive larger things (like utterances) and infer smaller things (down to the feature). And how could you ever tell? There is no coherent way to maintain the "perceive big, analyze down to the small" theory at least at the utterance-based end of speech. If we required an entire utterance in order to perceive language and had to figure out what the parts are based on that primary percept, you would predict that people had no idea what was said until the utterance was complete, at which point they can proceed to break up the utterance. That is patently absurd, and while it's not as obvious for English because of the relative shortness of words, it is also absurd for words as the basic unit of perception. You don't have to have experienced the entire word in order to know what sounds and parts of the word you're heard: you can cut off various amounts of the "allomorphy" and still be aware that the person has said "allo(m(or(ph(y))))". It's untenable to say that people directly perceive morphemes. We can't even decide where the morpheme breaks are, and the determination that the substring xyz is a morpheme may not be decidable until the end of the word. Morphemes are like underlying representations: they are the product of relatively sophisticated computation, involving a lot of analysis so that you can unify surface [mi,mu,mɨ,my] as realizations of a single morpheme. Morphemes, words and utterances have in common the fact that they are defined not in terms of what you hear, but state a relationship between what you hear and how you use it (e.g. "is that a complete sentence? is that a well-formed word? can you assign a semantic / grammatical function to that much of the sound input?"). The physical thing that is presented to people for decoding is sound, not function, but if you assume that people perceive words (etc) and discover sound properties from analysis of what they've perceived, you have to explain, on what basis can they know the function of something that they haven't yet discovered by analysis?

The three most credible and obvious candidates for the basic perceptual unit are sound-based things, the feature, the segment, and the syllable; the fourth, non-obvious but credible, is the demi-syllable (basically, a stretch of vocalic and consonantal material which isn't necessarily a whole syllable). Theoretical considerations favor the feature as the basic unit of sound perception: there are maybe 2 dozen features, and parsing on the basis of features is most economical. However, the relationship between features and the raw data is extremely complicated, to the point that one would have to smuggle a lot of inferential mechanism into your theory of perception.

Segments are reasonably credible, but less economical: there might be dozens or even a couple hundred segments (in Khoisan languages with over-loaded segment inventories). Syllables are very non-economical: the number of distinct units that would be perceived under a syllable-as-percept theory is usually big and may be gargantuan, depending on clustering possibilities and inventory size in a language.

But constructing an empirical argument for a particular choice is non-trivial. "Categorical perception" tests establish that in English (and a few other languages), speakers are sensitive to the narrow range of variation of certain acoustic correlates of phoneme differences, especially voice onset time. By manipulating a stimulus, speakers will respond "pie" vs. "buy" based on the particular manipulation, because the task is to make a categorical choice (what word is it?). They could be primitively perceiving syllables, or they could be primitively perceiving segments, and the task really does not tell you which it is.

I've referred to what I hold to be the basic unit of perception as "the segment", so as to avoid the definitional problem of phones vs. phonemes. They are both instances of the genus "segment", where phone is viewed from a perspective that is closer to the physical input, and phoneme is viewed from the higher-level grammatical perspective of function in a language. English has a number of voiceless velar phones – [k kʰ kʷ kʷʰ kʲ kʲʰ k˺] – and one voiceless velar phoneme, [k]. I do not believe that there is enough empirical evidence to support a claim that people perceive phones and analyze that into phonemes, or vice versa, so I don't make a claim. One "resolution" of the matter is to say that by definition, people perceive speech in terms of phonemes, and therefore you use (currently nonexistent) knowledge of the primitive units of perception as your definition of "phoneme".

  • Interesting.. I agree with you regarding perceiving sentences at utterance level, would not be possible, given the reason you describe. Why is phonemes is not a valid candidate for being a feature used for detection, if the problem with morphemes is the number of combinations, is that then solved using phonemes?... I guess I am misunderstanding the term "economical"? Commented Sep 4, 2017 at 11:39
  • About the categorical perception.. You are right... I didn't even think of it.. It could be that they changed the "feature" that altered the phonemes, but if it was perceived a phoneme, syllable or something else is us unknown.. So what you basically saying is no body knowns in what "chunk size" it is being perceived, but logically it would make sense that it in small bites.. but the bites has not been defined, or at least one close based on the experiment able to trigger the categorical perception.? Commented Sep 4, 2017 at 11:39
  • Why would perception at word level be absurd? Commented Sep 4, 2017 at 12:05
  • What determine the phoneme and morpheme divide lingustically? Commented Sep 4, 2017 at 12:09
  • Mostly addressed, but I don't understand the last question. Are you asking "how do you identify a sub-sequence of an utterance as an X" where X could be phoneme or morpheme (and the alternatives are "less than / more than an X")?
    – user6726
    Commented Sep 4, 2017 at 15:21

'Recognition' works at multiple levels, but (at the level you want) what we recognize are 'allophones'. These are sometimes similar to syllables, and early alphabets were syllabic. The muscle changes used to produce allophones allow representation as phonemes. Converting allophones to phonemes is only needed when spelling with a [semi-]phonetic alphabet.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.