7

What are the steps invoked in producing a dictionary?

I am primarily interested in understanding the role software plays in the production process.

Obviously a corpus for the language is first produced and this corpus needs to find a reasonable balance between technical terms, fiction, slang etc.

Then I'm guessing the size of the dictionary needs to be decided on and consequently the number of words it should contain. Then I'm presuming the most common words from the corpus make it into the dictionary and the rest are ignored.

But what happens then? Suppose I need to make the entry in the dictionary for the word "of" and I have 20,000 entries for this word in the corpus. How do I proceed? How do I keep track of the senses covered?

  • That seems to be the way Longmans does it, for instance, once they have their corpus. The OED corpus is great, but untagged. – jlawler Oct 6 '13 at 23:49
  • I've written an answer... is it what you were after? – Gaston Ümlaut Oct 8 '13 at 23:21
11

This is a complex topic but here's an attempt at an answer. My background is in describing and documenting relatively small, endangered languages, so I'll describe how it's done in that situation. I'll assume you have computers, software, internet, native speaking informants, recording equipment, and many years to spend on the work.

Firstly, recorded texts are needed, which are transcribed typically using software such as ELAN, and with much help from the native-speaking assistant. The resulting transcript should be time-aligned and have the following annotations at a minimum, each on its own tier:

*a phonemic reperesentation of the text
*a gloss (a very brief definition)
*a free translation

This is known as an interlinear gloss and is a standard analytical approach in linguistics. Here's an example (Western Desert language, Pama-Nyungan):

ngayulu  ngurra -kutu ya -nku
1.sg.ERG home -ALL    go -FUT
'I'm going homewards'

Another commonly included tier is a part of speech description for each lexical item (presupposing an analysis of the language).

Next this data would be imported into a tool such as Toolbox, FLEx or TshwanaLex. These tools then construct a database of the lexical items. The entry for each item will include the part of speech assignment, the gloss, a fuller definition, and can include many other pieces of information about the entry. These tools typically also enable creating a KWIC (key word in context) concordance which facilitates analysis of the text to discover the full range of meanings and uses of an item.

This 'lexical database' will typically be formatted using FOSF (Field-Oriented Standard Format), aka backslash codes, or at least will enable export/import using FOSF. FOSF consists of a backslash '\' combined with an arbitrary alphanumeric string, followed by the information it categorises. A fairly typical if basic entry would look like this (entry for the word 'papa', Western Desert language):

\lx papa
\pos noun
\def dog
\syn maliki
\xv papangku wati patjarnu
\xe the dog bit the man
\so JS

The codes: \lx = headword (this would also indicate start of a new entry), \pos = part of speech, \def = definition, \syn = synonym, \xv = vernacular example, \xe = english gloss of example, \so = source.

Entries such as this one can have a much more complex structure, including multiple senses. Each sense subentry would begin with a line such as \se1 (for the first sense), followed by its own \def, \syn, etc lines within the entry, as appropriate.

There is a variety of software that can take a regularly-structured FOSF lexical database and convert it into a nicely formatted dictionary. One easy to use software to do this that is currently popular is Lexique Pro, although ToolBox and FLEx can also do this conversion.

| improve this answer | |
  • 1
    Thanks for your great answer! I'm still unsure about how words with multiple meanings are dealt with. Take for example the word set: "set fire", "set the table", "set on the table", "game set match", "set time limit", "set in stone", "cement has set", "set dancing", "the sun set", "set a price", "set my watch", "set it up". How does the producer of the dictionary establish all these individual meanings? If the corpus contains 100,000 sentences with "set", its not like they can go through every single one by hand and cross check it with the 30 other meanings they have for "set". – Baz Oct 9 '13 at 7:28
  • @Baz: Feel free to ask less broad questions about very specific situations. Stack Exchange does much better with specific questions and discourages overly broad questions. You were lucky to get such a good answer to this one. – hippietrail Oct 9 '13 at 8:12
  • Hi @Baz that's exactly what you have to do, go through every occurrence of the form and work out the meanings (this is where the KWIC concordance is useful, but there are other tools). If the meanings are distinct enough (eg different parts of speech) then they may be different words that happen to have the same form. Otherwise they could simply be different senses of the one word. As I suggest in my answer, this work takes decades to do well. – Gaston Ümlaut Oct 9 '13 at 12:01
  • How does it cost to make a dictionary? Which process costs the most? – Ooker Jun 27 '16 at 15:07
  • @Ooker It's impossible to say how much it costs, it will be different for every language the main cost is probably in the amount of human hours required. – Gaston Ümlaut Jun 29 '16 at 5:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.