My question is about the current situation in machine translation. I am aware about two main approaches to machine translation. One which is based in a strong way on linguistic theory and another which is mainly statistical. Is this distinction correct? Is it the case that statistical methods with rather little Linguistics are at present most powerful? What are the relative strengths of the two approaches and to what extent is it possible to merge them.
7
-
5I think you're asking too many different questions, making the answer very broad. But let's see what others think too, first. :)– AlenannoCommented Sep 21, 2011 at 9:42
-
No I agree with Alenanno. @Gil please choose your favourite question and ask just that. We can deal with each other question separately for MT is a vast topic.– hippietrailCommented Sep 21, 2011 at 9:49
-
2please split the question to separate ones. At this point it is in danger of down voting or close voting– Louis RhysCommented Sep 21, 2011 at 11:12
-
I left the second question. Since the answers relate to the for related original questions here they are briefly: The first asked about the general situation in machine translation, the third was about statistical aspect of the modern theory of Linguistics? And the fourth was about foundational/philosophical discussions of MT more recent than Bar Hillel famous critique. 4) I recall a critique by Yehoshua Bar-Hillel regarding machine translation from fifty years ago. Are his comments still relevant? Are there further theoretical/fundational works on machine translation?– Gil KalaiCommented Sep 21, 2011 at 14:57
-
@Gil: the question about Bar-Hillel is interesting but hard to tell without a reference. If you find something that we can look at, please post as a separate question.– MitchCommented Sep 21, 2011 at 16:40
|
Show 2 more comments
4 Answers
Yes, there are two main approaches to machine translation. Rules-based (as in babelfish) and statistics-based (as in google translate).
I don't think that statistical methods are ipso facto better than rules-based methods in all things. You get different types of errors, as you'll see if you try to translate the same text to the same language with both engines linked earlier. It'll depend on the recipients what types of errors still leave the text understandable. I use Babelfish when I need translation-help for English -> German because I consider the errors of Google's translation service to be worse than those of Babelfish, for the kind of texts I need to translate.
Rules-based approaches takes time, money and trained personnel to make and test the rules. Statistics-based approaches needs large equivalent* corpora for each pair of languages it can translate to and from. Sometimes the former is easier to arrange than the latter, because of for instance copyright. For languages with few speakers rules-based is the only possibility since there exist no suffciently large corpora. That rules out statistical translation for most of the languages on this planet.
* Equivalent corpora: same genre of text. Corporas that have the same texts in different languages is even better but there aren't enough of those.
Statistical approaches certainly produce better results on certain standardized tests for languages and language pairs where there is a lot of data. Statistical approaches are less successful for languages and language pairs where there are not huge corpora.
It also depends if your goal is to do better machine translation or to better understand language in a psychologically plausible way.
I remember going to an ACL/COLING conference after being away from linguistics for over a decade and being shocked to discover all these people boasting about the f-scores they achieved "without using any linguistic knowledge".
-
7A famous quote in machine translation is "Every time I fire a linguist, the performance of our speech recognition system goes up." apparently due to speech recognition pioneer Fred Jelinek. Commented Sep 21, 2011 at 22:58
-
Not exactly in machine translation but in the related field of morphosyntax recognition we have noted that
- Statistical models are more adaptive, i.e. can be used for many different tasks with not much alteration
- Rule-based models are irrelevant for oral corpora
- On written corpora they have close f-scores, specifically on chunking and lexeme recognition statistical models have a good precision and rule-based models a good recall.
The conclusion being that the most efficient approach is using both : first use a rule-based model, then use its results as data for the statistical model.
Which paradigm is the more accurate or more adequate can not be answered in general. It may depend on what should be achieved since there are different advantages and disadvantages to both systems.
Many commercial approaches use a so called hybrid system which means that there is both a statistic and a rule-based component (or even an example-based component).
There are some advantages to the statistical approach, e.g. coping better with "irregularities" or better coverage of phrasemes, and some advantages to the rule-based approach, e.g. the abilitiy of syntactic validation. In hybrid systems like the Pangloss or the Verbmobil-system both components are parallel or in order with the goal to adjust both advantages for a better product.
