2

Given recorded speech with known text, is there an automated way to find the start/end time points of each sentence within the recording?

Google Cloud Speech API does not appear to return any time point metadata. SPPAS seems like it ought to do this, but based on the docs, it seems to only tokenize words, not sentences.

3

When the text is known (i.e., transcribed), segmentation (even on word level) can be done automatically.

Bayerisches Archiv für Sprachsignale (BAS), a CLARIN centre in Germany, offers a web service named WebMAUS for this purpose. Some tutorial on how to use WebMAUS can be found on TeLeMaCo, a Teaching and Learning Materials Collection provided by Universität des Saarlandes (another CLARIN centre).

| improve this answer | |
0

One approach would be to look at the volume (amplitude) of the audio signal at each point in time. If the signal falls below some configured level (accounting for any noise in the recording), treat it as silence. Then, if the silence occurs for at least a configured duration (e.g. 10ms) mark it as actual silence with the start and end times.

That will not give you the start/end of each sentence, but of each utterance (i.e. when the speaker decided to pause). This will tend to correlate to phrase-terminal (comma, etc.) and sentence-terminal (full stop, etc.), but not necessarily. For example, a speaker may pause before a conjunction ("and", etc.).

To provide better results, you need to perform full speech recognition on the audio with the text as a reference to match against. A simplistic approach here would be to match a few words around the sentence terminal, but you need to be careful to avoid word sets that could occur within the sentence itself.

Speech recognition is complex, and is harder to do when you don't know what is being said. This is because of accent differences, like the don-dawn merger in some American English acccents, suprasegmental features changing phonemes between words such as with the linked r sound (e.g. in "China and Taiwan"), and changes due to normal/informal speech.

Due to these problems, knowing what text is meant to be spoken can be used as a hint to the speech recognising algorithm to help guide its internal models (e.g. recognising 'atom' instead of 'Adam' for an American English speaker). This is still complex, because speakers may rephrase parts of the text, repeat some words or phrases, or introduce words such as 'er' and 'um'.

I don't know what software is available to provide the above functionality.

| improve this answer | |
  • How does this answer the original question about segmentation? – jk - Reinstate Monica Jul 25 '17 at 10:04
  • @jknappen I was giving an explanation of how to implement something like what BAS provide, not just listing "company X provide this functionality", while covering some of the issues likely to be encountered with this. – reece Jul 25 '17 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.