This might be a long shot but it will try it anyway?...

I am currently working on developing a speech recognition application, capable of detecting utterances consisting of the words yes/no.

A utterance could be of sort "yes no yes yes yes no yes" spoken at a normal speed.

I am currently detecting the words yes, no and silence - so its a 3-class problems.

I recently became interested in phonology, and the possibility of detecting words at a phone level could be cool, but a bit confused on what the words yes, no would be at a phone level?

Is phonetic division a subjective things us humans do, or are there general baseline rules for how words are divided into phonemes?

  • Take a look at the Kaldi toolkit, which is very easy to use, and which is excellently supported. It includes a very basic example of yes/no recognition (the directory egs/yesno). Commented Jul 2, 2017 at 0:38
  • @kkm I am currently using that exact example. Problem is that they aren't phonecticly but more at a word level. Plus I am using CNN for decoding with spectogram, and not the standard MFCC features. Commented Jul 2, 2017 at 9:54

2 Answers 2


Human speech is noisy, and speech recognition must be able to find patterns in the noise.

Phones have a series of articulatory attributes: places and manners of articulation, tongue shape, etc.; which cause voice resonance and distortion. All of those variables are continuous, and a continuous change in one of these parameters produces a continuous change in the produced phone. There's a spectrum of sounds between [a] and [i], for example.

Different languages make different distinctions between sounds, and some attributes are more important than others. For example, the distinction between voiced stops and voiced fricatives is significant in English but not in Spanish; similarly, the distinction between plain and aspirated among voiceless stops is significant in Hindi but not in English. So yes, it's a subjective distinction.

Also, not every English speaker pronounces yes and no the same way. There's a general pattern that makes a yes or no word recognizable, and a system could be trained to recognize these distinctions.

On its most basic form, you only need to be able to recognize "yes" and "no" from everything else, including ambient noise and other words. In fact, this might be a much easier problem to solve than a general speech recognition system, since you don't need to tell "no" from "know" (a context-dependent distinction) or lone "yes" from "YESterday".

You still need to be able to recognize the "yes" and "no" said speakers other than yourself, and that will require training your system with a decent amount of speech recordings.

  • Yes.. You absolutely right. My problem is very isolated, I am currently only using one speaker with one accent, and none or some context dependency is being used at a word level (unigram).. Still keeping it isolated to one language like english, would be a good place to start. My idea was to incrementally grow it from this base form and see how it improves. About the word pronounces, are you talking in regards to accent or something else?... If not, I would interested to know how phonetics dictionaries such CMUdict is available? Commented Jun 29, 2017 at 14:53
  • 1
    CMU is a phonemic, not a phonetic, dictionary, as are most pronunciation dictionaries, unless they are specialized to the point they list several dialects. In general, the convention is that IPA symbols are put within [square brackets] for phonetic transcriptions, and /slashes/ for phonemic ones. "Phonology" refers, in a narrow sense, to the way phonemes map to phones in a given language. In my opinion you should learn about this basic difference (you will find a few resources by searching for "phonemic vs phonetic" on a search engine) before venturing into further research on the subject.
    – LjL
    Commented Jun 29, 2017 at 15:37

I suspect you are not using the expression "phone level" in its technical sense. Also "detecting words at the phone level" isn't meaningful, but perhaps you intend "detecting words, given a waveform-to-phone conversion". To clarify, a "phone" is a reduction of the continuous acoustic waveform to percepually-based units (i.e. a narrow phonetic transcription). The words in question could be [jɛs] and [nɜʊ], but that really depends on dialect and details of a given token (e.g. [jɛs n̥ɜʊ] could occur). The premise of such a conversion is that there is a language-independent mapping from acoustic property to surface categorization: see this q&a for discussion). However, that premise turns out to be an ideal rather than a reality, at least currently.

The process of getting from continuous waveforms to ordered discrete unit is not subjective (it is based on real world facts), but we don't understand those facts well. Segmentation is a poorly-understood objective reality. There are numerous "baseline rules" in the phonetic literature over the past 70 years.

In order to "detect" a word based on a conversion to phones (not phonemes, which is where phonology comes in) you would need a list of all of the phone realizations of a given word, and that would be a pretty huge task, even using just one speaker. If you manage to solve the continous-to-discrete conversion problem, you could gather a large corpus of tokens of the words of interest, run that through your converter and (manually) tag instances of "no" and "yes"; that would give you a list of attested phone sequences representing the words in question, though not necessarily all of the possible realizations.

  • Thanks.. Yes I am seeking to divide them into the discrete unit.. I am bit careful about the usage of the terms phonemes and phone, as I am still a bit unsure on how i should differentiate both, seems like the internet isn't clear on that. but.. Since i am keeping the problem as isolated as possible, one person, same language, same dialect and so on, I don't see why it should be an issue, i am currently somewhat able to detect yes/no/silence from the given example utterances. Commented Jul 1, 2017 at 9:25
  • Because speakers are not consistent in how variable their segments are. A single speaker in a single recording session will have many different surface realizations in saying "yes" more than once.
    – user6726
    Commented Jul 1, 2017 at 14:23

Not the answer you're looking for? Browse other questions tagged or ask your own question.