As an English speaker, I've had very little experience with retroflex consonants, but have recently come upon their use in Polish, and am having difficulty hearing them as particularly distinct from non-retroflex consonants at similar articulation points. In particular, I've learned that the glyph <ż> indicates the [ʐ] phoneme, but I have enormous difficulty hearing any distinction from [ʒ]. Whenever I have this impression of any sounds, my first assumption is that the distinction simply lies somewhere I'm not used to listening for, but any suggestions on what aspects of the sound I should focus on would be helpful.
On the other hand, I do notice a fairly significant change to any surrounding phonemes, especially when they are vowels. For example, if I just make the sound sequences [aʐa] followed by [aʒa] I don't notice much difference in the consonantal part of the sound, but the [a] sounds in the former are very, very noticeably retroflexed. My initial assumption is that this effect is probably more a result of my relative clumsiness and slowness in moving my tongue into retroflex position; on the other hand, it also seems possible that the primary phonemic distinction might simply involve surrounding phonemes to a greater degree than in other types of consonants. Is this off base?