In this paper: Word Translation Without Parallel Data researchers from FaceBook found that using a matrix and adversarial training the word embedding in one language can be transformed into(or aligned with) that in another language linearly, then I wonder why it is possible? Any related theory in linguistics on this alignment?
Add a comment
|
This is really a neat question, and I enjoyed considering it while reading the paper.
Recall that word embeddings are a form of manifold learning. As such, this method presumes that communications across languages have similar semiotic manifolds. On one hand, this is quite a remarkable and beautiful result to behold; especially something so easily executed. Meanwhile, it shouldn't come as much of a surprise, given the overall similarities in experiential existence between one language-speaking people and another. A great boon to this form of training would be documents which have translations. If it were not for these, such a technique might not perform quite as well, but for most applications they're no problem; beneficial, in fact. Anyway, the linearity of a transformation between two language embeddings is proportional to the similarity of the respective semiotic manifolds.
