Making a balanced word list systematically using COCA and CMUdict

Question

I am trying to find words in English that mirror the conditioning environments for spirantization of /b, d, g/ in Spanish. I am balancing for proceeding phone, lexical stress, word-internal vs. word-medial etc. I'm holding the following phone constant (to /a/).

I want to be able to search CMUdict for words that have the the exact string of phones I need.

For example, I want to be able to search for a word that contains a word-medial token of /b/ that is preceded by /s/, part of the onset of a stressed syllable, and followed by /a/

I also want to limit this search to nouns. Like by using .[n*] in COCA. And also to integrate frequency (COCA).

I can do orthographic searches on the COCA, but I can't search for the actual phoneme pronounced which is why I like searching on CMU dict. On CMUddict I can also search for stress,but I can't do wildcard searches to get word-medial vs. word-initial tokens. I also can't control for frequency. I have been opening the CMU dict in chrome and doing searchers on the .txt file (using Arpabet)

I guess what I want to do is combine the best of both worlds. I was thinking about generating lists in the COCA and then importing them into CMU dict/ARPABET, but I'm not sure to do this. I'm open for any suggestions. Thanks!!!!

Answer 1

You should be able to do the phone environment searches you describe on the CMUdict. Instead of using a web browser, you need something with regex support (a good text editor). I would recommend something like Sublime Text as your text editor.

Searches for word-medial vs. word-initial are simple for a regex beginner:

'  (.+?AA1.+)' #this matches words with a non-initial, non-final AA1
'  (AA1.*)' #this matches words with an initial AA1

A complex one like "contains a word-medial token of /b/ that is preceded by /s/, part of the onset of a stressed syllable, and followed by /a/" may be difficult to do in one step (though perhaps easier to do in several steps). There are many good sites for help with regex and testing (http://www.regexr.com).

Combining the information of COCA and CMUDict is fairly simple if you're comfortable with scripting/programming. This operation is called a 'join' or 'merge' for tabular data. In Python, for example, you can do this manually by reading in the CMUDict (as a dict) and your COCA-based orthographic word list (as a list). In a loop, you look up the CMU match for each word. Your output will be a new dict (I'll call it 'outputDict').

for word in wordlist:
    outputDict[word] = CMUDict[word]

In R, you could simply read in both files as dataframes and do the appropriate join/merge operation. Using the plyr package, it looks like this (assuming wordlist has a column called 'orthographic':

join(wordlist, CMUDict, by = 'orthographic', type = "left", match = "first")

Other issues: CMUDict has a funny doublespace-delimited format, so be aware when you're working with it in Python, R, etc. The Python code provided (and similar approaches) will get confused if COCA has a word that CMUDict doesn't; the join approach won't break, but obviously will not find a phonetic match for those items.