Is there an accepted objective criterion of "grammaticality"?

Question

I'm not a linguist, but I took "linguistics 101" in college, and remain interested in the subject.

One aspect of the linguistics field that has baffled me for years is the fact that much of the discourse in it hinges on the ability to distinguish "grammatical" from "non-grammatical" sentences (or fragments thereof), but the details of this discrimination are never made clear.

(I used to think that these classifications were based on surveys of native speakers, but at some point a professional linguist told me that such a procedure is relatively uncommon.)

What happens when experts disagree on whether a particular sentence is grammatical or not?

I don't really expect a satisfying answer to this question here. Rather, I'm looking for pointers to articles that focus on the question of establishing objective/operational criteria for establishing grammaticality. (It's basically two big questions rolled into one: 1) is it important to have an objective criterion for labeling sentences grammatical or ungrammatical; and 2) if the answer of "1" is yes, what should this criterion be?)

(I'd prefer to start exploring this question by reading those authors who think that having such an objective standard would be a good thing.)

Answer 1

To answer your first question, it is vitally important if linguistics is going to be scientific (or at least rigorous) to have an objective way of dealing with grammaticality.

Your second question, however, is a bit more involved. I would personally split what a lot of people (even linguists) call "grammaticality" into several separate things, but mainly into acceptability, an empirical claim, as well as a variety of theoretical claims.

The first is acceptability. This is an empirical claim you can test either introspectively or experimentally by asking speakers if a sentence is acceptable or not. The reason why something is acceptable or unacceptable can vary. It could be that something is syntactically wrong with the sentence (for instance, *Herself hit Jean, where herself and Jean refer to the same person should be judged as unacceptable by all competent English speakers). But it could also be that something is semantically wrong with the sentence (for instance, Chomsky's example of !Colorless green ideas sleep furiously, which is grammatically fine, but semantically nonsensical). Or it could even be pragmatically wrong, where it is semantically and morphosyntactically okay (for instance, #My goldfish thinks I'm a lousy cook.).

However, there are associated theoretical claims--why a certain sentence is unacceptable. It could be morphosyntactically ungrammatical, which is a theoretical claim about what is morphosyntactically allowed or not in a given language. It could be semantically unacceptable, which means that the meaning just doesn't work. Or it could be pragmatically infelicitous; something about the wider context of an utterance means it just doesn't work. And, of course, none of these are mutually exclusive. While Chomsky's Colorless green ideas... is semantically unacceptable, for example, it is also pragmatically infelicitous. These arise not because of any sort of empirical test (though that's a good way to find them), but as a consequence of the linguistic systems a given language of a given speaker has in place.

One article more or less in the vein of what you're looking for, which deals with gathering acceptability judgments of morphosyntax, is Sprouse and Almeida 2011.

Answer 2

Let me try to provide an alternative answer to @limetom. It is a very good summary of the standard position in formal linguistics but does not answer all aspects of the question.

1. It is possible to have a highly scholarly linguistics that does not dwell on questions of grammaticality at all or only incidentally. Both corpus linguistics and construction grammar are more interested in typical usage than in introspective judgements of speakers about acceptability. Even functionalist approaches are not too concerned with grammaticality as such.

2. The other problem with grammaticality is that there is absolutely no objective measure of it. Chomsky famously copped out and said that any reasonably competent native speaker is just as good a judge of whether a sentence is grammatical or not. That is demonstrably false - one only needs to attend a graduate linguistics seminar to see that people's intuitions differ. In some cases, we can do surveys but even if we get 90% judgement of grammaticality, we still only discovered a variability in the input.

3. That is not to say that we cannot or should not be interested in speaker judgements about language. We want to look at how people correct themselves and each other. We want to see where they seem to be following some pattern in diverging from a standard or simply don't know or commit a slip of the tongue. But we can determine the standard patterns by analysis of actual usage - ie. the "scientific" method of observation. Introspection will still play a role in the analysis but the whole enterprise will not hinge on it.

In short, language is more than sentences (which are an object of writing rather than speech anyway). It is used for communication and self-expression. It is messy and complex. As part of it, people make judgements about acceptability and that is worth studying properly. But grammaticality itself is not a concept you need outside this narrow confine.

You also need grammaticality in some branches of applied linguistics:

For example, when you have an algorithm for generating sentences that is likely to spew out almost anything and you need a formal way of limiting the output of that algorithm. This is where you try to approximate native speaker patterns and turn to their intuitions about these.

Another example, is foreign language instruction. But here you have to construct a sort of inter-language that matches some key patterns of native speech but is not exactly the same. So you turn to generalized grammatical intuitions.

Finally, in contexts like copy editing or essay marking, native speakers turn to their intuitions to identify patterns that are unacceptable for some reason. Accidental slips, divergence from standard into dialect, lack of observance of register, etc. Much more is going on here than 'grammaticality' but often the people performing similar tasks will couch their efforts as avoiding things that are 'ungrammatical'.