0

This table lists many language codes for data sets, but I can't seem to find where the pair ze_zh or ze_en is defined:

None of the ISO code standards seem to list these pairs either.

Improve this question
1
  • 2
    This is probably an error, ask the source of the table for clarification.
    – Sir Cornflakes
    Mar 9, 2020 at 20:04

1 Answer 1

Reset to default
3

I think this means that some of their data comes from a bilingual Chinese / English source. zh means Chinese and en means English, so it's reasonable that they invented ze to mean a bilingual data source including Chinese and English.

Correct me if I'm wrong, but it looks like you got this table from the Open Subtitles Corpus? http://opus.nlpl.eu/OpenSubtitles-v2018.php

That points to the paper "OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles" by Lison and Tiedemman published at LREC 2016. http://www.lrec-conf.org/proceedings/lrec2016/pdf/947_Paper.pdf

The paper doesn't use language codes, but it has this quote:

The dataset also includes bilingual Chinese-English subtitles, which are subtitles displaying two languages at once, one per line (Zhang et al., 2014). These bilingual subtitles are split in their two constituent languages during the conversion.

I don't know why they don't just label these as en and zh, but once you look into it more, I'm sure it will make sense.

Improve this answer
2
  • Thanks. That looks to be exactly what the codes designate.
    – Brendan Martin
    Mar 10, 2020 at 15:44
  • Brendan, If you figure out more about how they work, feel free to improve this answer.
    – Jetpack
    Mar 10, 2020 at 18:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.