Can the Hunspell dictionary format capture things beyond prefixes and suffixes?

Question

I know this is mostly a technical / data question, but it is majorly about linguistics too and I don't think this is really up the alley of a lot of software engineers, so I am asking here.

So the Hunspell dictionaries are the foundation of modern spell-checking, and used by pretty much every browser (Chrome, Firefox, etc.), many applications (OpenOffice, Adobe products, maybe more), and several operating systems (MacOS, etc.). It is a very weird format though, here's the official docs which might help explain what's possible with it.

A hunspell dictionary is 2 files, .dic (list of words, with mapping to possible affixes) and .aff ("affixes", list how they work, but also has much more functionality). Here is the English hunspell .dic and .aff. Here is a snippet of the .dic:

accommodate/XGNDS
accommodating/Y
accommodation/M
accompanied/U
accompaniment/MS
accompanist/SM
accompany/DSG
accomplice/SM
accomplish/DSLG
accomplished/U
accomplishment/MS

First it is the word, then a slash, then the affix directives. So here is a snippet of the .aff file (that file is actually quite small at under 200 lines):

SFX N Y 3
SFX N   e     ion        e
SFX N   y     ication    y 
SFX N   0     en         [^ey] 

SFX X Y 3
SFX X   e     ions       e
SFX X   y     ications   y
SFX X   0     ens        [^ey]

The SFX (suffix) format is n+1 lines:

• first line: 1 2 3
  1. ID (name of the affix class)
  2. Cross product (permission to combine prefixes and suffixes). Possible values: Y (yes) or N (no)
  3. Line count of the following rules.
• n+1 lines:
  1. ID
  2. stripping characters from beginning (at prefix rules) or end (at suffix rules) of the word
  3. affix (optionally with flags of continuation classes, separated by a slash)
  4. condition.

The Sanskrit .dic and .aff (1000 lines), for example, are much more complex than the English one it seems. Hunspell seems to be able to handle Sanskrit sandhi just fine. For example, we have this rule:

SFX 397 Y 1
SFX 397 म ्देताम् म

So that is saying, there is 1 rule for ID 397, where we strip म from the end of the previous word, and add ्देताम्, assuming the previous word ends in म.

So Hunspell can handle situations where you basically do:

Apply this suffix to the previous word if the previous word ends with X, and remove Y from the previous word.

My question at a super high level is basically, can Hunspell handle every situation of how to combine affixes across every language, or are there cases where it breaks down?

To narrow the question down and take it out of the clouds a little, my more specific question is, can Hunspell handle situations like infixes or "circumfixes" of things like that? Like Turkish adjective intensifications:

• siyah ("black") → simsiyah ("pitch black")
• güzel ("pretty") → güpgüzel ("very pretty")
• temiz ("clean") → tertemiz ("clean as a pin")
• katı ("hard") → kaskatı ("hard as a rock")

Is it possible to define a rule for that in Hunspell? Or other situations more complex than Sanskrit?

Here it says:

Main features of Hunspell spell checker and morphological analyzer:

• Support complex compoundings (for example, Hungarian and German)
• Support language specific features (for example, special casing of Azeri and Turkish dotted i, or German sharp s)
• Handle conditional affixes, circumfixes, fogemorphemes, forbidden words, pseudoroots and homonyms.

So I assume it does, but if so, what is an example of these advanced cases (specifically around *fixes)?

Answer 1

My question at a super high level is basically, can Hunspell handle every situation of how to combine affixes across every language, or are there cases where it breaks down?

Hunspell was developed as a spell checker for Hungarian, so of course, there is much more focus on suffixation, and affixation generally, than to other morphological processes, which are not that productive in Hungarian. And also, being a computer program, concatenating strings is much easier than modifying a string in place, as would be necessary to handle Germanic strong verbs, the Semitic transfix systems, the infixes of Malay etc. etc.

Since you mentioned the sandhi processes of Sanskrit, these involve modifying a segment (usually a consonant) at a morpheme boundary, so it's equivalent to saying "if the word ends in X, then take the X off and stick Y on instead". So that's programmatically a simple operation for a computer program to do as well.

So here is a snippet of the .aff file (that file is actually quite small at under 200 lines)

It is not so surprising that English, a language with relatively few suffixes/prefixes, will have a small .aff file. By contrast, a Czech .aff has almost 3000 lines!

To narrow the question down and take it out of the clouds a little, my more specific question is, can Hunspell handle situations like infixes or "circumfixes" of things like that? Like Turkish adjective intensifications:

Your examples from Turkish, "simsiyah", "güpgüzel" and so on, look like reduplication to me, with the added complication that an arbitrary consontal segment is inserted between the two thingies. What other language famously does that? Why, Indonesian of course!

So let's have a look at a hunspell .dic file for Indonesian, which will need to describe exactly that process. That file has the lines:

ganda-berganda
ganda-ganda/B0Mu
ganda/B0DkM0MkMuPaRa

to describe the fact that the word ganda may reduplicate, but in that case must have one of the strings - or -ber inserted between the two thingies. It does not use hunspell's morphological rules to accomplish this; it uses separate dictionary entries. In other words, ganda, ganda-ganda and ganda-berganda are all treated as completely separate lemmas, just like Uganda and propaganda.

So there you have it. Hunspell's morphological rules does not usefully support the "situations like infixes or "circumfixes"" you're asking about, but this can be made up for by putting more dictionary entries in.