How to build a robust transliteration scheme across languages?

Question

So I am trying to imagine building a transliterator across languages that takes any language and converts it into IPA or some less-detailed equivalent (like a Romanization). I am thinking about English because it seems like a very hard example. English spelling often times has nothing to do with pronunciation, and you often times need to understand the meaning of the entire expression just to pronounce a word correctly. For example, "I created a tear" could mean "teer" or "tair" depending on what your story is about.

But some other languages (at least from looking at Wikipedia orthography tables) like Sanskrit seem to have strict rules for everything, so you start with consonants and a few vowels, and combine them up to make words. Then every word seems like it can be pronounced purely from the rules of this orthography, without regard to the meaning of the word, or it's placement in an exprssion.

But I don't know that for sure.

My question is, in order to build a robust transliterator across languages, what would be the bare minimum requirements?

• Do you end up needing a dictionary/list of vocabulary words, along with their IPA pronunciation, to do any "realistic" transliteration? That is, it ends up you toss the grammar/orthography rules out the window (they are more guidelines) and instead must use word-by-word pronunciation lists to do it well.
• Or which languages can you be just fine with only working off rules of the orthography/pronunciation? And which ones will you not be fine with (like English)?

Another aspect that might broaden the scope of the question too much, but I thought I'd throw it out there. Somehow we're able to speak with different "accents" (dialects?) and yet still understand each other. Like British vs. American English accents. The orthography is the same for all intents and purposes, but the IPA is different. So that sort of throws a wrench into the mix, you also need to specify what dialect you are speaking in to tell the transliterator how to do the final output, I think.

At this point in my journey I want to throw my hands up in the air and give up saying "you just need a giant database of word-for-word pronunciations, and then to create a mental meaning model, to do transliteration correctly". Ugh, isn't there anything in between that and a direct orthography to pronunciation mapping?

• e.g. https://academic.oup.com/jss/article-abstract/61/1/157/2472188

Research

Transcription / Transliteration

Arabic

• http://corpus.quran.com/documentation/phonetic.jsp
• http://www.cjk.org/cjk/arabic/arabsam.htm

Japanese

• http://www.cjk.org/cjk/samples/jpdpress.htm

Tibetan

• http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/M2asr-delivery-phaseI#Tibetan

Am I going in the right direction...?

Pinyin and Hebrew, for example, seem to be pretty precise. You learn the orthography and you learn how to speak. Is that true? How about of other languages?

Answer 1

There is a major difference between transliteration and transcription. Transcription is a thing that field workers do when they listen to a language and write down what they hear, assuming some system of transcription (for example IPA, but it could an number of local scheme like APA, Finno-Ugric Alphabet, etc.). Transcription only works on speech: you cannot "transcribe" text. Transliteration, on the other hand, changes one writing system into another, and operates on character strings. You can (somewhat) easily transliterate Arabic, Ge'ez, Devanagari etc. into another alphabet, but that requires knowledge of the conventions for transliterating the source alphabet. And that in turn usually implies making a choice of conventions. There is a Semitic transliteration standard that can be applied to Ethio-Semitic languages, so if you like p-with-underdot, you can follow that convention.

There is some slight chance of devising a systems that maps Tigrinya spelling to IPA transcription, if you will settle for broad transcription that may not correspond to the actual speech of any person. This was actually done rather successfully for North Saami, by the folks at Giellatekno, who modeled orthography to pronunciation for a speaker (using "teacher dialect", not her own pronunciation).

The situation with English is a worst-case scenario which I think should not be your starting point. There are plenty of languages where the mapping from orthography to pronunciation is (apparently) reasonably straightforward, although it turns out that Turkish has more phonetic vowels that are generally acknowledge in the literature (the e/ɛ problem).

The primary meta-requirement is deciding what you actually want: transliteration, or transcription. If transliteration, which convention? if transcription, what level of linguistic concreteness? Transliteration is clearly the easiest task. If you want transcription, you have to have a realistic understanding of what the spelling system tells you. What is the narrow transcription of "pity" in English (trick question – you have to say something about dialect). How is North Saami golbma pronounced? (there are probably hundreds of pronunciations). If you do intensive research on pronunciation of an individual, you can probably come up with a system that maps spelling to reasonably-accurate transcription, for that individual (it will fail for the next individual, but you may be able to concoct variant rules for person 2). In the case of North Saami, you have to know the morphophonemics of the language and you need a somewhat richer dictionary (not just the typical "Nominative / infinitive" word list). However, the major lexical sources for NS do include the inflectional details. The reason why you need to know this is that the length of the vowel can be computed if you know what "kind" of stem the word is (abstract C-final, which acts different from similar-sounding V-final). But if you don't care about getting vowel length right, you need less information.

There is no general rule regarding what you need to know: you really have to study languages on a case by case basis.

If you want to be able to pronounce a language, you primarily want a transcription, though there is no current (or reasonable future) technology to get from an IPA transcription to a decent pronunciation (non-native but "close enough" from the native speaker perspective). Suppose you were to train on standard IPA pronunciations using the Esling, Ladefoged, House, Wells corpus. You could transliterate (Tigrinya) ምሕሳብ to mɨħɨsabɨ, do a crude mapping to phonemes to get [mɨħsab], then add more details about pronunciation to get [mɪ̆ħsab]. It is usually pronounced [mɪ̆ħsab̆p] (brief voiced transition at the beginning of final /b/, then devoicing). Any of [mɨħsab], [mɪ̆ħsab] or [mɪ̆ħsab̆p] should be recognizable, even when pronounced according to IPA standards. So it just depends on how good you want your pronunciation to be. If you want to avoid all phonetic characters, and you want to have good pronunciations, then you cannot learn how to pronounce Tigrinya without learning that alphabet and having a collection of audio recordings associated with each word in the dictionary. You might be recognized if you pronounce this as plain English "meSAB".

Answer 2

There were a lot of attempts on this, I just want to mention the Weltschrift or Weltalphabet by Johann Martin Schleyer, a polyglot and the inventor of the constructed language Volapük. Some glimpses of his work can be found in the digitised issuses of the jounal Sionharfe here.

The attempts have proven to be futile, and nowadays linguists use IPA as the alphabet of last resort.