Speech and Language Processing without Representation of Language?

Question

While reading in the literature about text processing I found this scientific paper about text classification using artificial neural nets. The authors feed their convolutional neural network with just the letters from the text. The network's output is the classification so there is no internal representation of the text's content or syntax in the process.

Therefore to understand the neural net's decisions is a very hard task since visualizing filters and the neuron's activations is not as comprehensible as it is when working with images instead of texts.

I have two questions:

1. Are there any examples for solving other speech and language processing tasks without referring to language itself? For example a speech recognition system could map directly from my voice to computer commands on my phone without converting my speech to some internal representation of language.

2. Are there any solutions for the task of visualizing/understanding the decision making of convolutional neural networks that work with texts especially when the input is just the text letter by letter?

Answer 1

To your question 1)

Your question assumes that neural networks have no representation. Fact is, that neural networks have a layout (of layers, convolutional weight schemes) that are setup before training starts, and it highly depends on this layout what can be learned in that particular NN and whether it can solve a certain task. During learning, the weights in these layers are trained, and the overall configuration of weights again obtains a structure that allows to predict what is learned. This structure can be seen as a representation. Provocatively asked: the human brain is a neural network, so do you think humans cannot represent anything in their head? It is true that we cannot visualize the representation easily (see question 2).

Apart from this comment, I would say that the new version of Google Translate is probably the best example of a system "without representation" that applies CNN on a large scale in a productive system.

The research paper is here: Wu, Y., Schuster, M., Chen, Z., Le, Q. V, Norouzi, M., Macherey, W., ..., Dean, J. (2016). Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv:1609.08144 https://arxiv.org/abs/1609.08144

To your question 2)

Scientists are trying to visualize the decisions done in a neural network. This is especially important due to new laws that will require AI systems to give an account of how a decision was made. So far only logical systems provide a way to explain their decisions. If neural networks and other systems that do not use an explicit symbolic representation shall find their way into daily life, they will need to be able to explain their reasoning, or we will only be able to use logic programming and other rule-based systems like ontologies.

One recent work about explaining the reasoning of a neural network, already cited quite a bit, is `Abbeel, P., Chen, X., Duan, Y., Houthooft, R., Schulman, J., & Sutskever, I. (2016). InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. NIPS.' http://papers.nips.cc/paper/6399-infogan-interpretable-representation-learning-by-information-maximizing-generative-adversarial-nets.pdf

Answer 2

Well, even a character based neural network (CNN) does not only take the letters of a text into account, but also the their order. So, a text is not just reduced to a bag of characters.

However, a CNN is completely agnostic of morphemes, words, sentences, syntax trees and other linguistic representations, indeed. So it has some internal representation of the text, but this representation is probably very far from what we'd call a "linguistic representation" or a "representation of language".

To your question 1.

CNNs are a very hot research topic right now, and applications are sought for everywhere, but there is not much usable yet.

To your question 2.

The answer to this question is a plain no. The internal space of neural network (counting in hundreds of dimensions) is very hard to visualise, when it is visualisable at all.