How to search Penn TreeBank for arbitrary patterns?

Question

What is a general way to parse the trees in the Penn TreeBank for arbitrary patterns, e.g. perhaps I want all trees which have a plural subject and whose sister VP has a modal verb daughter.

Another example would be finding all trees which have a VP headed by some verb and which have a daughter PP headed by to. This isn't even a subtree match--it's only matching an incomplete part of the tree.

This doesn't seem very easy, because the trees are not regular structures but are presumably context-free (or even context-sensitive).

I thought about converting the trees to strings, but regex obviously doesn't not work for this. Regex is useful for finding whole (sub)trees, but I wish to find patterns among different phrases/nodes in the tree.

Answer 1

Your question is tagged for NLTK, but if you're free to switch, Stanford NLP has a product called Tregex which does exactly this kind of tree search.

You might also be able to shell out to the Tregex binary, which has a fairly useful CLI as well. See documentation on TregexPattern#main for more.

You can find some introductory documentation here.

(Disclaimer: I work in the Stanford NLP Group and have contributed to the Tregex project.)

Answer 2

There are quite a few tools which can do this.

For example: Tregex mentioned by Jon Gauthier.

I know a few others:

• PML-TQ https://ufal.mff.cuni.cz/pmltq/
• Tundra https://weblicht.sfs.uni-tuebingen.de/tundra-beta
• Iness search

Among them, I know Tundra best since I used to work in the group who created it. Tundra has a very nice UI based on bootstrap, as well as a cool tree visualizer based on D3. It uses the well-known tiger search language and supports searching both constituency trees and dependency trees. Thanks to the highly efficient indexing scheme it employs, Tundra can handle large-scale Treebanks, such as automatically parsed Wikipedia.