NLP text classification approach for smart devices

Question

As a project, I have been trying to understand the low-level heuristic approach from the perspective of a device attempting to behave smart, like Alexa and the likes

Generally speaking, it starts with STT, then text classification, PoS/NER, IR and finally TTS. (maybe some additional steps, but I am generally stating here)

I can't clearly understand the text classification part properly.

How is the device able to come to a conclusion what the subject matter of the text is?

Various text classification heavily rely on the fact there would be large corpora to train with. And this makes sense, particularly for if the application is news categorisation, for example.

But here we are talking about spoken language. If the sentence is Tell me the weather, there's only so much corpus you can generate for the variation in that simple statement. And still, find some other way to ask for the weather.

I don't think for each category there can be a large datasets of statements which would help to make the device clearly distinguish between commands.

So the question
1. Is the classification purely rule-based, statistical or hybrid?
2. If either of these, any of these approach would introduce ambiguity. How to deal with that if building one and eventually approach near unambiguity? Since more categories (or skillsets) would mean more similar statements.

PS - The hardware is on raspi, with Google STT, naive bayes for classification, nlp-compromise for NER, various APIs, and finally eSpeak.

PPS - Sure there is SVM, deep learning, RNN and more cool topics. But I don't see them do a classifier job only for this purpose, even if I rig a GPU externally to talk to it. But honestly I am not clear of what could be a reliable method for classification

Answer 1

Alexa is far less smart than we're led to believe. The real short answer to your question is: Those classifications are man-made, i.e. defined by developers. In Alexa, they're called a "Skill", and skills set the parameters for the syntax and sense it attributes to all words associated with this skill by mapping a skill with a content catalog, to which one must also introduce a long list of sample sentences covering a wide range of question/utterance possibilities associated with said skill.

Example: "Weather" is considered a skill. In "What's today's weather?" versus "What's tomorrow's weather", Alexa automatically determines that the category is "weather" and fetches the mapped sense for "today" and "tomorrow" linked to "weather" to establish which Accuweather page it fetches. Underneath it all though is Amazon's machine learning platform and advanced deep learning functionalities driving the automatic speech recognition (ASR) (the stt engine), and natural language understanding (NLU) to recognize the intent of the text, which in this case is in deducing that one is asking a question about the category "weather", as well as stochastic processes that sets an interpretation to all utterance variants not included as sample sentences.

That's the general gist of it, but it really is as simple as that: categories are defined; a word(s) must be assigned as a trigger(s) for a category and, of course, these words must be as specific as possible, and their trigger linked to a set of grammatical functions, usually when the word occurs as the head of a subject NP.