I've noticed that most NLP systems I can easily investigate on the Internet don't handle ambiguity at all, or only handle within components of the pipeline and not between components.

For instance it seems typical that text segmentation, morphological analysis, POS tagging, parsing, semantic analysis, etc do not pass ambiguity information along to the next stage, even if one or more individual stage might have some awareness of ambiguity.

I'm looking for any NLP system I can play with (free or online or open source, etc) that embraces the concept of ambiguity through every stage.

The reason I'm interested in this is that an early stage can easily choose an interpretation which a later stage finds less acceptable and reject an interpretation that could be accepted by later stages.

  • 1
    It makes a huge difference if intonation, stress, and rhythm information is available. That's what's used in natural language to ignore ambiguity and nudge the perception to the "obvious" choice. Because they don't encode this data, written utterances (which are presupposed by most NLP systems) just doesn't have the information to handle ambiguity, with the result that every written sentence is multiply ambiguous. We forget this because of our mind's ear, if we're fluent readers.
    – jlawler
    Sep 25 '14 at 14:38
  • Oh yes I'm aware of the "pervasive ambiguity" problem. I'm interested in speech recognition too but I'm concentrating on text because it has a lower barrier of entry. I'm expecting there must be some systems or standards for passing parse forests or DAGs between stages. Some stages may be able to prune some ambiguities from earlier stages but I'm aware that even at the end of the pipeline many may remain. Sep 25 '14 at 14:56
  • 1
    There are many POS taggers, parsers, semantic analyzers, etc. that deal with ambiguous data. In most of those cases, the algorithms are built to add up the weightages of the various components, and then provide only the 'best' solution; because for most purposes, providing 10 solutions rather than just one makes the tool useless. Many of these tools provide ways to enable the 'nbest' option. If you want an easy-to-use system, look for Stanford Parser. If you're in the mood for some heavy lifting, look for the delph-in toolchain. These are just two examples. I think I could come up with twenty.
    – prash
    Sep 25 '14 at 16:28
  • @prash: The more I dig into these topics I find that passing n-best / k-best is not what I'm looking for. Some papers I've been reading through clarify the difference between passing that limited info and passing the complete ambiguity info. Sep 30 '14 at 2:10
  • @hippietrail: The n-best systems mostly account for syntactic ambiguity, and to some extent semantic ambiguity. Can you please expand the question to include examples of ambiguities you are looking for?
    – prash
    Sep 30 '14 at 11:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.