The vowel analysis in Chomsky and Halle's (1968) SPE is said to be overly abstract. What are the main issues phonologists have with this abstractness?
There was a feeling at the time that the distance between underlying and surface forms in their analysis was too great, and there must be some absolute limit on how different the two levels of representation could be (say, measured in number of features changed, or whether underlying feature matrices or sequences of matrices had to be a subset of observed matrices). The attempt to articulate any such formal restrictions was quite unsuccessful. With a bit more rational analysis, we came to understand that the issue was not really about the qualitative distance between representations (a.k.a. abstractness), but rather it was about other methodological assumptions that SPE made.
The primary factors that motivated their abstract analysis were (a) their aesthetic desire for symmetry, (b) the wish to formally connect all possibly related word forms, and (c) the desire (prevalent at the time) to capture every generalization out there, even ones that deserve to roam free. Since their formal theory of rules was extremely device-rich, a lot of things would be possible then, that were deemed to be formally impossible in later developments of phonology. (Note how I say nothing about the current state...).
For instance, they wanted to assume that various and variety have a common root, which led them to a certain abstract analysis of the i ~ ai alternation, and similar facts drawn from word pairs like derive ~ derivative and the ɪ ~ ai alternation led to them positing underlying forms that reflect Chaucerian pronunciation. Then they conclude that the English vowel system really has no underlying diphthongs, except for ɔi, so that makes way for a symmetry argument that "coin" has to be underlyingly /kœ:n/ (because of the rules they set up, that derivation comes essentially for free, and now there are no diphthongs).
SPE was an integrated package deal, and great effort was put into accounting for every little fact without having any exceptions, thus a stress puzzle could force a different and more abstract analysis of [ow] (veto ← /ve:tɔ/, motto ← /mɔto/). In their analysis, [ˈraitʃəs] 'righteous' derives from /rixt-i-ɔs/ and [dʒəˈræf] 'giraffe' from /giræffe/, and so on. Oddly and uncharacteristically, they also note 2 pages later (p. 152) that "giraffe" could also come from /ǰVrāf/ which is only two steps away from saying that the surface form is the same as the underlying form. The general reaction was that there was something wrong, and "abstractness" was blamed.
My position is that blame should be placed not on abstractness, but on gratuitous assumptions about morphology esp. the question of whether historically related forms are to be reduced to common roots and affixes in all cases; on their idealizing assumptions about underlying representations; and on the fact that they were trying to operate without an autonomous theory of acquisition (that is, the structure of grammatical theory was supposed to automatically account for all of acquisition).
The most relevant-seeming tests for how naive speakers understand the structure of their language concern rhyming in poetry and how words are deliberately distorted in secret languages (such as pig-latin). These tests do not so far seem to support the SPE analysis. (You could look at articles by David Stampe and Paul Kiparsky for discussion along these lines.)