As a project, I have been trying to understand the low-level heuristic approach from the perspective of a device attempting to behave smart, like Alexa and the likes
Generally speaking, it starts with STT, then text classification, PoS/NER, IR and finally TTS. (maybe some additional steps, but I am generally stating here)
I can't clearly understand the text classification part properly.
How is the device able to come to a conclusion what the subject matter of the text is?
Various text classification heavily rely on the fact there would be large corpora to train with. And this makes sense, particularly for if the application is news categorisation, for example.
But here we are talking about spoken language. If the sentence is
Tell me the weather, there's only so much corpus you can generate for the variation in that simple statement. And still, find some other way to ask for the weather.
I don't think for each category there can be a large datasets of statements which would help to make the device clearly distinguish between commands.
So the question
1. Is the classification purely rule-based, statistical or hybrid?
2. If either of these, any of these approach would introduce ambiguity. How to deal with that if building one and eventually approach near unambiguity? Since more categories (or skillsets) would mean more similar statements.
PS - The hardware is on raspi, with Google STT, naive bayes for classification, nlp-compromise for NER, various APIs, and finally eSpeak.
PPS - Sure there is SVM, deep learning, RNN and more cool topics. But I don't see them do a classifier job only for this purpose, even if I rig a GPU externally to talk to it. But honestly I am not clear of what could be a reliable method for classification