1

For a student job i'm creating a neural network-based method of determining the probability that two written names are referring to the same person (e.g. what is the probability that kelly m. refers to kely möreson in a simplified example).

To train my network, i'm creating a simulation that generates populations with family structures and corresponding names. The simulation works, but the difficult part is to "corrupt" the data in a realistic way.

If one of the names from my simulation is "kris walton taylor", i'd like this entry to be corrupted in forms like "chris walton t.", "kris w.", "chris wolton tailor", "kris talor" and so on. Basically, all the different forms in which people would write the name based on hearing it.

I am using a tool from an academic institute in germany to convert the names into their most likely ipa representation for a given language: https://clarin.phonetik.uni-muenchen.de/BASWebServices/interface/Grapheme2Phoneme

Now, i'd like to use the list of ipa representations to convert them back into different ways of writing the name. In a perhaps naive attempt i've tried to create a library of all the different letters that can be associated with each ipa symbol and use that to randomly generate corrupted names, but this has given me completely unrealistic results (for example, kris was converted to kʀɪs in ipa, and when corrupted it gave me results like qrrees or ckryc).

I realize that the alphabetical letters associated with ipa symbols have a certain probability of being "chosen" based on the surrounding letters. This seems like a enormous task to codify from scratch however, so i'd like to check if there are any existing algorithms that attempt this?

To be clear: i'm not looking for an algorithm that gives me the most likely spelling. I'm looking for an algorithm that produces random spellings with a realistic probability.

For example:

Given the ipa string "bʀʊk", i'd like the algorithm to give me 6/9 times "brooke", 2/9 times "brook", and 1/9 times "bruke". This is a completely random and not academically proven example, i just want to illustrate the type of results that i'm looking for.

abbreviations and other types of corruption are not required, i already have working algorithms for those.

Does anyone know of a source that can help me along? It doesn't need to be an algorithm, just a set of written rules are already better than what i have now. I can turn the rules into a working algorithm myself.

Either a set of rules for english or spanish would be massively helpful, but i'm happy with any other language also!

3
  • The trouble with names is that they don’t necessarily play by the normal rules. As a British surname, /brʊk/ might be spelt Bantroulcque for all you know (I don’t think that’s an actual name, but algorithmically, you’ve no way of knowing that – /ˈbiːtʃəm/ being Beauchamp or /ˈfanʃɔː/ being Featherstonehaugh are no less outré), and a /krɪs/ may actually be a Khryss these days, particularly in younger generations. – Janus Bahs Jacquet Jun 13 at 12:56
  • 1
    What @janus said. But in addition, you need to figure in syllable and word structure to make things work better for you. For example the digraph < ck > never occurs at the beginning of a syllable nor, thus, at the beginning of a word. Similarly, the letter < c > is not pronounced /s/ at the end of words [or rather word-final /s/ is never spelled with < c >]. As a rule of thumb /s/ is only represented by < c > if the < c > is followed by < y, e > or <i>. This kind of consideration will clean things up a tiny little bit. But probably not as much as you'd like/need. – Araucaria - him Jun 14 at 15:03
  • Thank you @Araucaria-hehim, what i'm looking for is a complete set of these rules so that i don't have to come up with them myself. – JadaLovelace Jun 14 at 15:26
1

As Janus Bahs Jacquet mentions in a comment, specifically looking at names makes this much more difficult. Names come from a wide variety of languages, and are Anglicized to a wide variety of different extents. This also makes it extremely difficult to figure out their pronunciation in the first place: when you see a surname like "Beauchamp" or "Zhang", how do you decide if you're using the original (French/Mandarin) pronunciation or the Anglicized one?

The best overall solution will probably be a huge lookup table, if you can find a list of common names and their pronunciations.

The best algorithmic solution will probably be to take some algorithm for converting spelling to pronunciation (this one is fairly straightforward; there are more complex ones out there too), turn it into something like an FST, then invert that (using something like FOMA). You'll end up with a nondeterministic transducer that converts a pronunciation in IPA to (almost) all possible pure-English spellings. This will work well for names that follow modern English spelling rules, like Smith and Brown; it'll work much less well for archaic names (which are extremely common) and foreign ones.

2
  • Thank you for mentioning FST and FOMA. I have some studying to do! – JadaLovelace Jun 14 at 12:59
  • @JadaLovelace FOMA doesn't have the best documentation or the easiest interface (for example, it tends to crash rather than giving error messages), but I've found it extremely useful. – Draconis Jun 14 at 15:59
2

In order to have any hope of succeeding, you need to first limit the scope of the data. For example the name pronounced [ʃidʒɪnpʰɪŋ] is usually spelled Xi Jinping, and [ɛnvɹ̩ hodʒə] is spelled Enver Hoxha. You might decide that you want to rule out names that are not "English", but then you need a way of deciding if a name is "English". This is going to be really problematic with last names like "McDonald", "Jensen", as well as first names like "Donald" and "Thomas". A list of higher-frequency names is called for ("Hoxha" won't make the cut). You will end up with "Wang" and "Wong" and probably "Patel".

The simplest solution, if you have such a list of names, is to give the attested pronunciations for each name, or at least the most frequent ones. As far as I know, "Wong" is pronounced [wɔŋ] and "Wang" is [wæŋ] or possibly [wɔŋ]. You probably don't have to take into account [rɛɪf] as a pronunciation of "Ralph", though because of frequency effects you might want to add a special entry for Ralph Fiennes, which itself would have the alternative spelling "Rafe". That is, what you need is data, not rules, because spelling and pronunciation of names is the least rule-governed aspect of English.

If you are absolutely committed to conversion rules rather than data-lookup, you should just implement standard spelling rules of English, knowing that in the domain of names, you will fail very often. Given a spelling, there is a range of probable pronunciations, and given a pronunciation, there is a range of probable spellings. Chris and Kris is well-attested, and you will probably have to forego variants like Khry(s)s, Chriss or Kriss.

1
  • The scope of my data is limited in the sense that i have the spellings of the names in my database already. i just need to corrupt them according to contemporarily common spelling behaviour. For my research i'm using a database of most common names from the US census bureau, so names with a "unique" spelling are not part of my simulation and the scope of my research does not go that far. The furthest i'm going is to look into both spanish and english names. Thanks for your insight. I'm going to search for name databases that contain alternative spellings for each name. – JadaLovelace Jun 13 at 17:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.