A so-called round-trip translation is not a reliable signal that the translation worked well, for a few reasons.
Noisy Parallel Corpora Are Bi-Directional
Machine translation systems train on parallel corpora, which are often noisy. For example, suppose the following noise occurs in the training data:
en: "This is a rare sentence." is: "Rænðöm rænðöm gíbberish."
Note that there is no sense of directionality here; the English-Icelandic system learns:
en: "This is a rare sentence." -> is: "Rænðöm rænðöm gíbberish."
while the Iceland-English system learns:
is: "Rænðöm rænðöm gíbberish." -> en: "This is a rare sentence."
In this simplified example, the output of the round-trip translation - trans('is', 'en', trans('en', 'is', x)) - will likely be x. The two mistakes cancel each other out. This happens more often than it would if translation mistakes were simply random.
Even Good Translation Is Lossy
A translation, and even a language itself, is lossy. Distinctions of meaning, gender, tu/vous, tense and aspect are not represented equivalently or at all in all languages. Ambiguities are also not necessarily represented.
Let us consider an example, with English as the intermediate language to make it clear:
es: "La profesora Le está esperando."
We can legitimately translate this into English as:
en: "The teacher is waiting for you."
[Note: We lose the gender of the teacher, whether it is a schoolteacher or university professor, and the formality of you, and any choices about word order.]
Now we translate back into Spanish:
es: "Te espera el maestro."
That round-trip translation very different than the original, although the translation in both steps was correct. This dynamic applies to both human and machine translations.
Machine Translation Is Not Perfect
The lack of readability of the machine translation of Icelandic text written by a human is possibly a signal that the human did not use that machine translation system but did, in fact, translate by hand in some way. In fact, good idiomatic translations are more difficult for machine or human translators.
The last dynamic applies to evaluating a human translation by machine translating it into some language you know.