1

The performance of a machine translation system is highly impacted by the parallel corpus it is trained on. Therefore, as we all know, a good quality normalized and (ideally) noise-free corpus is essential. To know how a MT system is performing we have to actually train the model first and then test its performance on the test data set. This is sometimes time consuming due to the fact that the data sets are usually huge. For my thesis I have 5 different versions of my parallel corpus. If I precisely want to measure the performance using BLEU score or some other metrics I need to train the model 5 times and that will take a lot of my time. Therefore, I was wondering if there is any way to measure the quality of a parallel corpus beforehand? I have found this article which is kind of doing what I am looking for, however, I cannot cite it for my thesis as a reference since it is not a published work.

4
  • Just to be clear, you're wondering if there's any mathematical (or rather principled) manner of judging a single language corpus, or a two-language corpus that is supposed to be used for translation? If the latter, well, that's the problem you're trying to solve already with MT, isn't it?
    – Mitch
    Commented Oct 24, 2017 at 21:03
  • 2
    Also, yes, you can cite that article, you just have to say that it is 'unpublished manuscript' (see your bibTeX style recommendations for exactly how to put this in a citation)
    – Mitch
    Commented Oct 24, 2017 at 21:05
  • 1
    Thanks for your comment. It's the former actually. Once I have prepared a parallel corpus, say English to Spanish, I want to 'guess' or 'predict' whether the English (or Spanish) corpus I have in my hand is going to give me good quality translation. Commented Oct 24, 2017 at 22:43
  • Parallel corpuses. Both have to be a corpus.
    – Lambie
    Commented Jun 11 at 16:56

1 Answer 1

0

There are theories, but generally training machine translation is more of a practical endeavour of trial and error.

What is the actual use case and evaluation methodology?

For example, just raising BLEU score is a different game than making a production system for hospitals more robust to noisy input. They may even be at odds.

Assuming you have a clear goal, here are some theories to try:

  • Size
  • Relevance
  • Coverage
  • Balance
  • Quality

Quality generally means target quality, not source quality. Low-quality source may actually be a good thing.

Just keep in mind that a formatting aspect, like segmentation or HTML escaping, may have more impact than the actual wording.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.