I'm trying to research methods of identifying or pattern matching names of characters in a novel or a general body of text, but so far my search has been unsuccessful, since "character" refers to those of the alphabet in context with "pattern match". Some ideas I've had:

  • Find any set of repeated words in Title Case
    • This causes a problem with "Was Phileas Fogg rich?" by matching "Was" however
  • Find transitive verbs that are unique to people, such as: "Jack said", "replied Jack"
    • This won't find characters mentioned in passing
up vote 9 down vote accepted

This is the natural language processing (NLP) task of named entity recognition.

As you have observed, creating a bunch of heuristics will get you a long way, but not the whole way. Following modern trends in NLP, the best-performing approaches in this problem learn statistical models from data, rather than functioning off hand-written rules.

An adequate statistical model will automatically learn some of the rules that you would otherwise have to define by hand, for example, "a word following a verb in title case is probably a name".

The NLP group at Stanford has a state-of-the-art named entity recogniser based on conditional random fields (a machine learning technique). The models it includes are for the more general problem of recognising People, Locations and Organisations, but you can filter the output for People only.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.