I hear a lot of people talk about how "new" linguistics is, or how "small" it is compared to other fields. Pāṇini studied grammar in the 4th century BC. Surely it didn't take until recent history to study it scientifically?
This is a great question. Not that it matters all that much, but it's always good to periodically revisit the directions one's discipline has taken.
First, the problem is that there's linguistics and then there's study of aspects of language for a particular purpose.
Lexicography, pedagogic grammar, philosophy of language - they all have a long tradition in the West. You could almost say that applied linguistics came first.
One of the earliest significant works of linguistic theory was the Port Royal grammar of 1660 but it can in no way be seen as even a pre-cursor of modern linguistics. Its importance was retrospectively recognised by Chomsky but it had almost no direct influence on the development of the discipline.
The true starting point of linguistics as a separate discipline is generally identified in the work of William Jones (more than 100 years after the Port Royal grammar). Jones recognized commonalities between several Indo-European languages and thus started what is still recognizable as modern philology. From him, we can trace the developments of the following century almost in a direct line.
It would be another 100 years before we could point to something that looks like truly modern linguistics. By this I mean, an empirical research paradigm aimed at discovering the principles behind the workings of individual languages, their building blocks and language as a general phenomenon. Some names that stand out along the way are Wilhelm von Humboldt and Hermann Paul but it's not until the work of people like Otto Jespersen, Ferdinand de Saussure, Jan Baudouin de Courtenay and Vilém Mathesius at the start of the 20th century that we get output that we can read and still find linguistic affinities with (note: philologists can go all the way to Jones). de Saussure is by far the most famous but mostly through the efforts of his students. The competence/performance dichotomy can be directly traced to him (Chomsky's inventing history by pointing to the Port Royal grammar as his true antecedent. It was really the structuralists.) And, of course, it is only slightly later that Edward Sapir and Leonard Bloomfield contributed their significant syntheses that echo in the work of linguists to this day.
We should also not neglect the developments in affiliated disciplines which have been developing along side (if often slightly behind) general linguistics. Phonetics and phonology, psycholinguistics, and even philosophy of language each have their own interesting histories and intertwining but separate interests from those of linguistics. Then there are those of subdisciplines like sociolinguistics, contact linguistics, discourse analysis, corpus linguistics, etc. which each also have trajectories that are worth pursuing most of them not really starting until the 1950s. In many people's mind, linguistics is identified with generative linguistics but that is only one of the many subdisciplines of the field whose importance was artificially inflated at least in part due to US defense funding of AI research (see Frederick Newmeyer's histories on this).
Sadly Pāṇini's is always only mentioned as a footnote. Yet, his influence on all the Sanskrit scholars must have been significant. When you compare his meticulous treatment of Sanskrit grammar (including phonology) from at least 400 BCE with the meager output of European grammarians since the days of Plato, you cannot be in awe. Arguably the work of Indian grammarians provided models of best empirical practice for European students of Indo European languages but it is hard to estimate exactly what impact it's had on linguistics as we know it today. But it is without question the greatest work of empirical and theoretical linguistic inquiry prior to about the mid 1800s.
Finally, let's talk about the question of "scientific" study of language. Chomsky and his followers often cover up their embarrassing ignorance of the vast field of linguistics by dismissing anybody not in their formalist tradition as somehow not scientific enough. Whatever you think about Chomsky's own theory (and I think it is an impressive achievement if not really that much about the empirical phenomenon most people would describe as language), this is just pure and unadulterated nonsense. It's a rhetorical rather than an empirical device that is unfortunately all too common in academic discourse. But it is no less disreputable by its ubiquity.
Post Script: Many alternative perspectives could be offered on this subject. I have certainly focused on the work with which I am most familiar and to which I feel a great affinity. However, I suspect that for all its biases, that mine is fairly mainstream perspective on the history of linguistics. I would accept quibbles and corrections on almost every particular but the overall trajectory would probably remain the same. I wrote this from memory (relying on Wikipedia to check the spellings of names) influenced as much by my reading of the source materials as histories of linguistics I read and classes in the history of linguistics I took about 20 years ago. I spend a lot of time trying to find connections between old and new understandings of language but mostly in a rather unsystematic manner, so I took this opportunity to summarize some of my mental notes.
Modern syntax was first put on a scientific basis by Zelig Harris and Noam Chomsky: Harris with his development of formal models of phrase structure and Chomsky through his elaboration of Harris' techniques, by his clarifying the distinction between empirical study of syntax and language pedagogy, and by devising the first formalized model of the traditional idea that many language constructions involve movement of sentence parts to new positions. That last was, of course, transformational grammar.
In an historical account of our discipline, Pāṇini of course deserves a prominent role, but his grammar can't be accounted as right or wrong, except on accuracy of factual claims, because it is all about pedagogy. It gives a method for reciting the scriptures without error. While Chomsky made the crucial point that a scientific account needs to make predictions about expressions that are never problematic for human language learners.
I do not mean to imply that Chomsky's theories have been correct theories. As a matter of fact, I think that transformational grammar has been shown to be wrong. But this is a characteristic of scientific theories -- they can be shown to be wrong. It is confused to disparage Chomsky's place in developing modern linguistics because you happen not to be a follower of TG, or minimalism, or whatever Chomsky's very latest ideas are. The great thing about TG is that you can tell whether it's wrong. (It is, as Ross and others showed.)
No-one has cited a textbook yet. Linguistics: An Introduction to Linguistic Theory. p 5.
1.1 Panini to Chomsky and After
The interest in the nature of human language appears to have arisen when the human species evolved in the history of time. There is no culture that has left records that do not reveal either philosophical or practical concerns for this unique human characteristic. Different historical periods reveal different emphases and different goals although both interests have existed in parallel.
Egyptian surgeons were concerned with clinical questions; an Egyptian papyrus, dated ca. 1700 BCE, includes medical descriptions of language disorders following brain injury. The philosophers of ancient Greece, on the other hand, argued and debated questions dealing with the origin and the nature of language. Plato, writing between 427 and 348 bce, devoted his Cratylus Dialogue to linguistic issues of his day and Aristotle was concerned with language from both rhetorical and philosophical points of view.
The Greeks and the Romans also wrote grammars, and discussed the sounds of language and the structures of words and sentences. This interest continued through the medieval period and the renaissance in an unbroken thread to the present period.
Linguistic scholarship, however, was not confined to Europe; in India the Sanskrit language was the subject of detailed analysis as early as the twelfth century bce. Panini’s Sanskrit grammar dated ca. 500 BCE is still considered to be one of the greatest scholarly linguistic achievements. In addition, Chinese and Arabic scholars have all contributed to our understanding of human language.
The major efforts of the linguists of the nineteenth century were devoted to historical and comparative studies. Ferdinand de Saussure (1857–1913), a Swiss linguist in this tradition, turned his attention instead to the structural principles of language rather than to the ways in which languages change and develop, and in so doing, became a major influence on twentieth-century linguistics.
In Europe and America, linguists turned to descriptive synchronic studies of languages and to the development of empirical methods for their analysis. Scholars from different disciplines and with different interests turned their attention to the many aspects of language and language use. American linguists in the first half of the century included the anthropologist Edward Sapir (1884–1939), interested in the languages of the Americas, language and culture, and language in society, and Leonard Bloomfield (1887–1949), himself an historical and comparative linguist, as well as a major descriptive linguist who emerged as the most influential linguist in this period. Both Sapir and Bloomfield were also concerned with developing a general theory of language. Sapir was a ‘mentalist’ in that he believed that any viable linguistic theory must account for the mental representation of linguistic knowledge, its ‘psychological reality’; Bloomfield in his later years was a follower of behaviorism, which was the mainstream of psychological thought at the time, a view that precluded any concern for mental representation of language and, in fact, for the mind itself.
In Europe, Roman Jakobson (1896–1982), one of the founders of the Prague School of Linguistics, came to America in 1941 and contributed substantially to new developments in the field. His collaboration with Morris Halle and Gunnar Fant led to a theory of Distinctive Features in phonology, and Halle has remained one of the leading phonologists of the last decades. In England, phoneticians like Daniel Jones (1881–1967) and Henry Sweet (1845–1912) (the prototype for G. B. Shaw’s Henry Higgins) have had a lasting influence on the study of the sound systems of language.
In 1957 with the publication of Syntactic Structures, Noam Chomsky ushered in the era of generative grammar, a theory which has been referred to as creating a scientific revolution. This theory of grammar has developed in depth and breadth. It is concerned with the biological basis for the acquisition, representation and use of human language and the universal principles which constrain the class of all languages. It seeks to construct a scientific theory that is explicit and explanatory.
The chapters that follow are based to a great extent on the developments in linguistic theory that have occurred since the publication of Syntactic Structures in 1957 and Aspects of the Theory of Syntax in 1965. In subsequent years, Chomsky has continued to develop his theory in such major works as Remarks on Nominalization (1970), Conditions on Transformations (1973), Lectures on Government and Binding (1981), Barriers (1986), Principles and Parameters in Syntactic Theory (1981), and The Minimalist Program (1995).
In the following chapters, basic notions in these publications as well as many others in areas other than syntax are presented at an introductory and basic level.
The simple answer is that the field of linguistics as it is understood today begins in the 20th century, with deep roots in prior study of language. The practice of asking questions about language and how it works is much more ancient, as is the practice of asking questions about the mind and how it works and develops, but it is not until the 20th century that we start to see a systematic approach that links those questions, which in my view is what marks linguistics as a discipline today.
By way of analogy, the field of chemistry can be said to start in approximately the late eighteenth century. The practice of asking questions about the internal structure of matter or the interactions of different sorts of matter goes back much further, through alchemy and various informal knowledge systems (Roman concrete, for example) back to the Greek speculations, and so forth, but it is not until it the questions of internal structure and interactions become linked that modern chemistry begins.