Where to start if you want to do Chomsky style NLP?

Question

I am a computer science grad who has been fascinated by Chomsky's theory of language. I have been following his work and the others in his field. But I also want to try something on my own. The problem, I don't know where to start.

Briefly, I would like to work and contribute on NLP, but in the Chomsky's way and not the statistical one. I am aware of the the contribution of Statistical NLP and know the theory, tools and software around it.

What I am completely blank about is the research (in computer science) that is done on the line of Chomksy Linguistic theory.

PS: In case you want a refresher on what Chomsky's theory of Language is and how it differs from the Statistical one, below article are great starting point.

https://chomsky.info/20121101

http://norvig.com/chomsky.html

Any suggestions are appreciated!

Answer 1

I hadn't heard the term "statistical theory (of language)", but it seems to be a misnomer. I gather from your references that you take some data and use it to estimate the parameters of some statistical model. Model, not theory. We inherit our ideas about what empirical theories are like from the physical sciences, and a key property of those theories is that they are vulnerable to counterevidence. They can turn out to be wrong. Can a statistical theory of language turn out to be wrong? There don't seem to be any underlying ideas there that could fail. I think "model" is a better term than "theory" for such a framework.

On the other hand, transformational grammar, the theory that Chomsky proposed in, e.g., Aspects of the Theory of Syntax was evidently an empirical theory, since it did turn out to be wrong. I'll refer you to John Ross's dissertation, Constraints on Variables in Syntax, where the "constraints" are hypothetical, unformulated limits on the crucial variables in Chomsky's formalization of transformational theory. No one has proposed such constraints, explicitly, and it is probably impossible to do so.

There is no agreement in linguistics about how to describe the phenomena described by the Ross constraints, but my favorite candidate is the theory (based on Chomsky's context free grammar) described in Generalized Phrase Structure Grammar, by Gazdar, Klein, Pullum, & Sag.

Any of the three books that I've mentioned would be good things for you to look at.

Answer 2

Are you familiar with any of the HPSG or lexicalist approaches to NLP that came about in the 90s or anything? This is strictly non-chomsky, non-transformational but it at least would give you some kind of starting point with how NLP is done from more of a linguistics perspective. I'd suggest looking at ALE grammars, LinGo or anything like this. I am by no means an expert on Transformational grammar and was taught in a very anti-Chomsky environment but from what I gather there is good reason no successful attempts have been made at creating a computational model of transformational grammar. As alluded to by Greg Lee, essentially there arent hard rules to be coded and trying to come up with them might not even be possible. The HPSG stuff I brought up is a direct response to this. The "rules" in HPSG are easily represented in a computer and there have been large grammars written for quite a few languages. If you want to look at this seriously, I would suggest looking at the GPSG book suggested by Greg Lee and then also look at Bob Carpenters The Logic of Typed-Feature Structures. Carpenter's book is exactly what the title says and Typed-Feature structures are essentially how words/phrases get represented. I actually used all of this to write a mini-Mandarin grammar for a class in grad school. A little disclaimer: Im sure you are aware, but this territory starts to become a little murky in that a lot of this stuff has been abandoned to a large extent in favor of the statistical methods you mentioned. It definitely is still used but it is frankly hard to find resources for it. My personal theory is that a majority of serious efforts are done in the private industry and essentially amount to large-scale knowledge representation as a backend for other tasks. Also, in order to do this well you need to have an extremely strong grasp of the language you are working on. Syntax, semantics, pragmatics, and phonology all can play a role in your grammar and if you haven't spent a significant amount of time working on that language you could easily be creating a grammar that is totally misinformed.