How good is current machine translation (in ChatGPT4, or Claude3.5) at dissimilar languages, e.g. English and Chinese? Does it work perfectly now?

If not, can you give some examples of where they translated something from Chinese into English incorrectly?

1 Answer 1


Do they translate perfectly now?

No, not even for more similar language pairs like English : Spanish.*

Generative models are basically just at par with dedicated translation models.

In a professional workflow, roughly 50% of the sentences generated end up getting edited by a human.

Both types of models tend to fail exactly in the most unique and thus important bits of content.

After all, the Transformer model architecture at the core of genAI was first invented and launched for Google Translate back in 2017.

The main advantage of generative systems in practice is that they consider context across sentences.

Translating perfectly, or even just at par with humans, would require AGI, because in many cases common sense or meta context is required.

So if translation were a solved problem, then so would be countless other problems. Billions of people would no longer need to learn English to access medical information, research papers, business opportunities…

Unfortunately that is not yet the case, so billions of people cannot access most content nor share their own with the world.

* The big gap is really between language pairs with lots of data and language pairs without. English : Spanish and English : Chinese are similar on that dimension.

That said, if I were still working at Google Translate or on machine translation (instead of working at ModelFront on machine translation quality prediction) and obsessed with achieving human parity, I would focus on very similar language pairs, like Spanish : Portuguese, or Simplified Chinese : Traditional Chinese.

  • 2
    I like using LLMs to translate not only because they consider surrounding sentences, but in addition because I can provide some context myself or requirements for the translation (think things like using T/V, but Google Translate does that already, while you need an LLM for arbitrary requirements).
    – LjL
    Commented Jul 7 at 19:18
  • @LjL Right, it is easier to configure on the fly. Commented Jul 7 at 19:31
  • Transformers date back to 2017, but that's not a good argument, by itself. They got much better since then, by being much bigger, being trained on vastly more data, and some architectural improvements.
    – MWB
    Commented Jul 7 at 21:31
  • @MWB Yes, but the fundamental breakthrough since then was more about widening its application to more tasks than to furthering their performance for the one task for which we already it. What would be your explanation for why ChatGPT, which is truly impressive, hasn’t actually replaced (or outperformed) Google Translate, DeepL etc in practice in consumer or enterprise? Commented Jul 8 at 4:55
  • 1
    "What would be your explanation for why ChatGPT, which is truly impressive, hasn’t actually replaced (or outperformed) Google Translate, DeepL etc" - maybe because of inertia and user's belief that a dedicated tool is better. Also, more browser plugings. Google translate is absolutely awful but DeepL is descent. Also, an LLM can hallucinate, add something, depending on settings and length of the text, especially if it exceeds the context and it will not warn about difficulties in translation.
    – Anixx
    Commented Jul 9 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.