In just about every respect r behaves like the two glides: w and j. So Why exactly is it not classed as a glide? It is particularly its intrusive behaviour that makes me wonder why.
It depends on the language (note that there are many kinds of "r" and IPA has a wealth of r-like letters to indicate some of those differences). In very many languages, [r] or [ɾ] (etc.) is not treated as a glide, because it acts differently from glides. For example, onset clusters in Chinese can include glides, but not r. There is a common syllabification rule that syllables can end with glide plus consonant, but not liquid or nasal plus consonant, and r acts like a linquid rather than a glide. However English [ɹ] is rather different. In terms of features, r is usually [+consonantal] (which has to do with there being a constriction in the vocal tract smaller than a certain size), but English [ɹ] is very unconstricted and it is [-consonantal]. So it does in fact act like a glide, and is treated as a glide. The moral of the story is that not all rs are the same, and not all treatments of r are the same.
Why does it matter? If we start calling it a glide, does that predict it can be intrusive? How?
Historically, in Preliminaries to Speech Analysis, IIRC, an acoustic account was given for classifying r/l as both consonantal and vocalic, as compared to the glides, which have neither property. Something about having zeros in the spectrum and having periodic properties. I don't believe in acoustic features, so I have a hard time caring.