-2

The end goal of asking this question is to find a big (probably several hundred petabyte) data file expressing the meanings of words based on their interactions with other word in as many languages as available for some programming code being writing. However, 3 major problems arise when trying to obtain this meta-dictionary being looking. These three problems make this dictionary not just an ordinary synonym list, but rather a unspoken implication, literal context, and implied connotation based thesaurus.

Problem #1: Unspoken Implications
Let's compare the following 3 sentences.

1. How do I use a chair at a table?
2. How do I sit down at a table?
3. How do I sit down in a chair?

As seen in the 3 sentences above, it's quite easy for a human to understand that all 3 sentences are basically asking the same question. However, that is not the way a computer sees it. The way a computer would see it, sit, chair, and table are not direct synonyms, so all 3 sentences are not asking the same thing.
The solution for this is the thesaurus being looked for needs to contain a list of which word are implied in which contexts. For an example relating to the the above 3 sentences, this unspoken implication part of the thesaurus might contain an entry of technical information that can be understood in english as:

A person uses a chair to sit down at a table.
This example unspoken implications entry allows the computer to fill-in-the-blanks with the 3 example sentences listed at the beginning of this section. For example, when the computer cross references example sentence #1 with this thesaurus entry, it will think: the person in the thesaurus is the personal pronoun I, the use is use, the "chair" is chair, and the table is table. Thus, because there is only one remaining unfilled blank, it is safe to assume that this blank is filled with the default which in this case is the wordsit`. In this way, the above entry in the thesaurus would allow the computer to understand that the 3 example sentences at the start of this section are asking the same thing.

Problem #2: literal Context
I use the term "literal context" to describe words where their definition is based on other modifying/modified words. A very good example of this is the word nuclear which can mean a lot of completely different things based on the context. Nuclear can refer to the nucleas of a cell (ex. "nuclear envelope"), dangerous radioactive material (ex. "nuclear energy"), to immediate family (ex. "nuclear family"), to an ailment called "nuclear cataract," and even to extreme excitement (ex. "The crowd of people went nuclear"). In a case like as the word nuclear, a simple synonyms list can't just be used because the meaning changes based on the literal context. Thus, for this aspect, there needs to be a massive thesaurus to also list which words mean what when used in conjunction with which other words.

Problem #3: implied Context
Implied context is where the definition of words cannot be determined just based on the literal interpretation of which words surround it. Rather human intuition and common sense are needed to figure out what is ment. In this, we will examine the usage of the word "date" in 8 different contexts.

1. I asked her father for permission to have dates.
2. I asked the father for permission to have her dates.
3. I picked a date between the two females.
4. My girlfriend and I go to the store to have dates.
5. I asked her what today's date is.
6. I asked her when today's date is.
7. I dated her to college.
8. I dated the female to 536 B.C.


While in normal situations implied context is much less blunt, it still does make up an appreciable portion of what we speak. In example #1, dates is referring to the romantic phenomena. However, as seen in example #2, this assumption cannot be known through just having a gender-biased word ("she") that modifies or indirectly modifies the word "dates". Then, in #3, it is seen that while stating "I picked a date" would likely refer to the date fruit, when "between the two females" is added, date now refers to the romantic phenomenon. Further, in #4, it is seen that the meaning of "date" even depends upon the setting. Going to a store for dates would refer to the fruit, whereas going to a restaurant would refer to the romantic phenomenon. Then, as seen in #5 and #6, date can even flip-flop between romance and fruit just based on the type of pronoun used. Finally, as seen in #7 and #8, "date" can also depend on the time period. #7 refers to the romantic phenomenon, whereas #8 refers to the archaeological dating as in determining the time period of an artifact. Thus, since the implied diction can get so tricky, a very complex tangled web of conditional word relations would be necessary to figure this aspect of the problem out.

Conclusion
The overarching question is where can a find a gigantic list of the meanings of all words in all languages in all contexts that includes at least these kinds of interactions between words as listed above. Please note that it the list doesn't have to be perfect, it just has to be workable. For example, there may be some odd-ball idioms with an unanticipatable definition that are based on the context of the surrounding passage instead of the surrounding sentence. Bizarre lingual phenomena things like this would also be very helpful to know about, but are not a priority.

5
  • 3
    Your question spans across several areas of linguistics and computational linguistics. Could you narrow it down, or split it into multiple questions so that people will have something more manageable to answer?
    – prash
    Feb 23, 2018 at 20:07
  • 3
    You use 'all' a lot. I think that's asking too much. For one language (English) with most everyday words with some polysemy and some contexts, a good start for you would be WordNet.
    – Mitch
    Feb 23, 2018 at 21:00
  • WordNet is a good start (it's even in prologue), but I am looking for something a bit more grander, large-scale, and fully encompassing.
    – Jack G
    Feb 23, 2018 at 22:37
  • 1
    Even for one language, a product like this would be such a huge undertaking that there would be no commercial possibility of freely releasing it. Even to access the OED takes a subscription. And then to include dozens or hundreds of other languages? You're dreaming, unless you own half of Apple or Google.
    – curiousdannii
    Feb 24, 2018 at 14:07
  • 2
    But hey, maybe you could parse Wiktionary into a meaningful single database. That's somewhere to start at least. Good luck!
    – curiousdannii
    Feb 24, 2018 at 14:09

1 Answer 1

1

You have two input choices: 1. analyze dictionaries (wordnet, worknik or wiktionary), 2. use word embeddings (word2vec, glove, elmo).

Use this data with a WSD (word sense disambiguation) solution. Search github for WSD. Also, this is worth looking at: https://blog.openai.com/discovering-types-for-entity-disambiguation/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.