Automated methods to align text

Question

I have an assignment for my machine translation class which involves using the different language versions of websites as parallel corpora. The problem is the text isn't properly aligned.

For example:

http://www.ryanair.com/ie/terms-and-conditions

vs

http://www.ryanair.com/de/geschaeftsbedingungen#regulations-aircarrierinformation

According to textmechanic.co, the English version has 481 sentences / 807 lines and the German version has 533 sentences / 909 lines.

Does anybody know of any way to automate / verify the text aligning process? We have a tried writing our own Java program to match as many sentences as possible and throw away ones which don't match. However, we encountered errors at punctuation such as full stops for Mr. / Mrs., etc.

Does anybody have any experience dealing with this sort of situation?

Answer 1

You could use hunalign or one of the tools used by the open source parallel corpus (the main one, uplug, has several parts designed to align sentences).

I also read about CorporAl but I do not know if there is a stable version of it yet.