How to identify character names in a body of text?

Question

I'm trying to research methods of identifying or pattern matching names of characters in a novel or a general body of text, but so far my search has been unsuccessful, since "character" refers to those of the alphabet in context with "pattern match". Some ideas I've had:

• Find any set of repeated words in Title Case
  • This causes a problem with "Was Phileas Fogg rich?" by matching "Was" however
• Find transitive verbs that are unique to people, such as: "Jack said", "replied Jack"
  • This won't find characters mentioned in passing

Answer 1

This is the natural language processing (NLP) task of named entity recognition.

As you have observed, creating a bunch of heuristics will get you a long way, but not the whole way. Following modern trends in NLP, the best-performing approaches in this problem learn statistical models from data, rather than functioning off hand-written rules.

An adequate statistical model will automatically learn some of the rules that you would otherwise have to define by hand, for example, "a word following a verb in title case is probably a name".

The NLP group at Stanford has a state-of-the-art named entity recogniser based on conditional random fields (a machine learning technique). The models it includes are for the more general problem of recognising People, Locations and Organisations, but you can filter the output for People only.