All Questions
Tagged with machine-translation corpora
4 questions
1
vote
1
answer
77
views
Is there any theory for mathematically judging the quality of a parallel corpus before running it through a machine translation pipeline?
The performance of a machine translation system is highly impacted by the parallel corpus it is trained on. Therefore, as we all know, a good quality normalized and (ideally) noise-free corpus is ...
4
votes
2
answers
432
views
Probabilities for 2-grams are higher than 1-grams in arpa file produced by kenlm
I'm using the 1 billion word language corpus to build a model with 1 and 2-grams. When using the lmplz program that comes with kenlm, I noticed that the arpa file seems to have higher probabilities ...
- 141
2
votes
0
answers
59
views
Generalisations which a bi-gram probabilistic model might infer from a dataset
I have the following exam question for a machine translation course:
From my understanding, I assume the answer is looking for incorrect English grammar which get discovered by bi-grams.
So the ...
- 171
4
votes
1
answer
142
views
Automated methods to align text
I have an assignment for my machine translation class which involves using the different language versions of websites as parallel corpora. The problem is the text isn't properly aligned.
For example:...
- 171