concept/name extraction rules

Question

I'm doing an NLP research, trying to extract concepts/names from phrases, I need your help defining all rules for doing so, so i can feed them onto the computer.

example rules I observed:
if a preposition is followed by a Noun, then that noun is a concept

• ex:in toshka, it's always sunny
  Concept: toshka

if Verb, 3PS present is followed by Noun, then that Noun is a concept

-ex: how old is Ron ?
Concept: Ron

an so on, notice that i can't take all nouns blindly, consider this example:
when will the dark knight rises be released ?
the concept here is the dark knight rises, which is not a noun.

hope my question is clear, I need all language/grammar rules that you (as a human) use to identify entities.

Answer 1

I was thinking about your "The Dark Knight Rises" example and I was thinking "well, if it was properly capitalized then it's obvious that it's a title of something." Then I thought "but it wasn't capitalized and I still knew. How can I tell it's a title, even if I didn't have real world knowledge that it's a movie?"

One of the ways I can tell it's a title is because of the way it's embedded in sentence. On the surface it's an embedded CP but it is taking the place of a DP. That got me thinking about how syntax might help you.

I think you should read about the double object construction. The current view of double objects was first put forth in "On Double Object Construction" by Larson (1988). I can't find a full version online for free but these lecture notes that I found should give you the main ideas.

If you generate a syntax tree it should be easy to pick out the arguments you want. Although I have never used it before today, I've been playing around with the Standford Parser and it looks like you might be able to adapt it to your needs.

Incidentally, the Stanford Parser was able to identify "the dark knight rises" as an NP:

(ROOT
  (SBARQ
    (WHADVP (WRB when))
    (SQ (VBZ does)
      (NP (DT the) (JJ dark) (NN knight) (NNS rises))
      (VP (VB come)
        (PRT (RP out))))
    (. ?)))

Answer 2

You could also look into the ANTLR parser.