There is a difference between "Chomsky" and "modern linguistics". Chomsky is the founder of an influential school of modern linguistics, but there are other schools of modern linguistics. Chomsky's school further fragmented so that a number of arguably generative practitioners nevertheless disagree with Chomsky on some point and perhaps only accept the Chomskyan notion of an abstract mental grammar, and none of the formal details.
Focusing only on the portion of linguistics that is some version of generative grammar, the apt question would be, what direct influence does de Saussure have: who reads Course in General Linguistics? It is not generally required reading in this theory, but then Apects of the theory of syntax is also not required reading, nor is Lectures on government and binding, unless maybe you are going for a PhD in syntax in a strongly-Chomskyan program. Part of the answer is, linguists usually don't read very far back in the history of the field. "History of linguistics" is not usually a required subject.
There is a point in one's career where one is likely to read the classics, but getting a job and tenure does not require having read de Saussure (or Panini, or Aristotle, or Bopp, Sweet, Brugmann, Sapir, Bloomfield, Jakobson...).
As for "arbitrariness", there is a more basic issue that language is not entirely arbitrary (or "conventional"), though the phonological form of "signs" is conventional ("dog" is how we say it in English, "chien" is French, and so on). I don't know of any theory of linguistics that held that phonological form follows from some natural function. Since this is not a controversial point, it isn't widely discussed and therefore he is not widely quoted on this matter.
On the other hand, the distinction between langue and parole is given repeated lip-service, though via a translation to "competence" and "performance", and that is where Chomsky cites de Saussure the most.