I would like to parse a piece of text (Spanish if possible, otherwise English) and point each word in the text to its correct entry in a dictionary. For example:
I turned her down.
would point to the most common definitions of I and her while pointing "turned down" to the definition:
to refuse or reject (a person, request, etc.)
I am currently using the Stanford Nlp Pipeline via python, while the sentences I am using are coming from https://tatoeba.org/eng/. I am interested in any solutions which might help me to do this.
My ultimate aim is to estimate if a sentence contains only words which employ very common meanings. For example in the case of the sentence "I turned her down", all the words are very common but the specific meaning "to refuse or reject (a person, request, etc.)" is not at all common.
UPDATE
Just to clarify things further. If we take these three sentences:
I ran to the shop. I ran out of batteries. Please run that by me again.
We can see that only very common words are used, however the first sentence is also using the most common meanings associated with these words while the second two contain phrasal verbs. "run out" is somewhat common but "run by" is less so. I would like to be able to associate a higher score with the first sentence, with the last sentence getting the lowest score.
This is the nlp data I extract for the last sentence:
{
'index':2,
'basicDependencies':[
{
'dep':'ROOT',
'governor':0,
'governorGloss':'ROOT',
'dependent':1,
'dependentGloss':'Please'
},
{
'dep':'ccomp',
'governor':1,
'governorGloss':'Please',
'dependent':2,
'dependentGloss':'run'
},
{
'dep':'dobj',
'governor':2,
'governorGloss':'run',
'dependent':3,
'dependentGloss':'that'
},
{
'dep':'case',
'governor':5,
'governorGloss':'me',
'dependent':4,
'dependentGloss':'by'
},
{
'dep':'nmod',
'governor':2,
'governorGloss':'run',
'dependent':5,
'dependentGloss':'me'
},
{
'dep':'advmod',
'governor':5,
'governorGloss':'me',
'dependent':6,
'dependentGloss':'again'
},
{
'dep':'punct',
'governor':1,
'governorGloss':'Please',
'dependent':7,
'dependentGloss':'.'
}
],
'enhancedDependencies':[
{
'dep':'ROOT',
'governor':0,
'governorGloss':'ROOT',
'dependent':1,
'dependentGloss':'Please'
},
{
'dep':'ccomp',
'governor':1,
'governorGloss':'Please',
'dependent':2,
'dependentGloss':'run'
},
{
'dep':'dobj',
'governor':2,
'governorGloss':'run',
'dependent':3,
'dependentGloss':'that'
},
{
'dep':'case',
'governor':5,
'governorGloss':'me',
'dependent':4,
'dependentGloss':'by'
},
{
'dep':'nmod:by',
'governor':2,
'governorGloss':'run',
'dependent':5,
'dependentGloss':'me'
},
{
'dep':'advmod',
'governor':5,
'governorGloss':'me',
'dependent':6,
'dependentGloss':'again'
},
{
'dep':'punct',
'governor':1,
'governorGloss':'Please',
'dependent':7,
'dependentGloss':'.'
}
],
'enhancedPlusPlusDependencies':[
{
'dep':'ROOT',
'governor':0,
'governorGloss':'ROOT',
'dependent':1,
'dependentGloss':'Please'
},
{
'dep':'ccomp',
'governor':1,
'governorGloss':'Please',
'dependent':2,
'dependentGloss':'run'
},
{
'dep':'dobj',
'governor':2,
'governorGloss':'run',
'dependent':3,
'dependentGloss':'that'
},
{
'dep':'case',
'governor':5,
'governorGloss':'me',
'dependent':4,
'dependentGloss':'by'
},
{
'dep':'nmod:by',
'governor':2,
'governorGloss':'run',
'dependent':5,
'dependentGloss':'me'
},
{
'dep':'advmod',
'governor':5,
'governorGloss':'me',
'dependent':6,
'dependentGloss':'again'
},
{
'dep':'punct',
'governor':1,
'governorGloss':'Please',
'dependent':7,
'dependentGloss':'.'
}
],
'tokens':[
{
'index':1,
'word':'Please',
'originalText':'Please',
'lemma':'please',
'characterOffsetBegin':43,
'characterOffsetEnd':49,
'pos':'VB',
'before':'\n',
'after':' '
},
{
'index':2,
'word':'run',
'originalText':'run',
'lemma':'run',
'characterOffsetBegin':50,
'characterOffsetEnd':53,
'pos':'VB',
'before':' ',
'after':' '
},
{
'index':3,
'word':'that',
'originalText':'that',
'lemma':'that',
'characterOffsetBegin':54,
'characterOffsetEnd':58,
'pos':'DT',
'before':' ',
'after':' '
},
{
'index':4,
'word':'by',
'originalText':'by',
'lemma':'by',
'characterOffsetBegin':59,
'characterOffsetEnd':61,
'pos':'IN',
'before':' ',
'after':' '
},
{
'index':5,
'word':'me',
'originalText':'me',
'lemma':'I',
'characterOffsetBegin':62,
'characterOffsetEnd':64,
'pos':'PRP',
'before':' ',
'after':' '
},
{
'index':6,
'word':'again',
'originalText':'again',
'lemma':'again',
'characterOffsetBegin':65,
'characterOffsetEnd':70,
'pos':'RB',
'before':' ',
'after':''
},
{
'index':7,
'word':'.',
'originalText':'.',
'lemma':'.',
'characterOffsetBegin':70,
'characterOffsetEnd':71,
'pos':'.',
'before':'',
'after':'\n'
}
]
}
