I'll start by saying I'm not trained in formal linguistics. So I won't have the slightest qualms being told what I'm saying doesn't make too much sense.
I'm a native speaker of American English. I've studied German, Latin, French, and Japanese in school. I've been in Japan for almost two years and had worked for years to be ready to move before that. I now, speak, read, and write the language to a decent extent (failed the N1 by 4 points a few months ago). In general, I have not had any difficulty perceiving the sounds produced by Japanese as the same kana they do.
Recently, however, I've encountered a feature from a subset of Japanese speakers where they produce /rj-/ and I hear /j-/. For example, students introducing themselves Ryou or Ryuusuke [are Japanese characters allowed on linguistics.se?]. I only hear yo and yusuke even if they repeat it. If different Japanese native speakers say the same names, I clearly hear the /rj-/. The speakers who produce a sound I fail to hear are male native speakers of Japanese aged about 19 living in Hokkaido (and generally native to Hokkaido).
Again, this is only a subset even for people in the same sample group. I don't normally have difficulty hearing /r/ or /rj/ from other Japanese speakers and produce them in ways where they are perceived as correct by native speakers in Hokkaido.
Background for those not familar with Japanese: These are two mora openers produced by combing the symbols for "ri" and "yo" and "ri" and "yu" respectively.
(1) What does the subset of native Japanese speakers who produce a type of /rj-/ sound that I only hear as /j-/ do differently from other speakers? (2) Is this a known linguistic property?
I've gotten some helpful answers from the Japanese.se as to what might be going on that help explain why there ain't much sound there. Native Japanese correctly hear /rj-/ even though there are minimally different names (in the case ryusuke there is a name yusuke)
This wikipedia entry hints at what could be answer in mentioning a flexibility in how the /r/ sound in Japanese can be produced. But my random googling gives many different accounts of what /r/ in Japanese is ("an apical postalveolar flap undefined for laterality" vs retroflex flap)
Here's a sample of a speaker who produces it in a way that is hard for me to hear. I definitely hear the leading /r/ on his first try but don't perceive it on his second very well. Can someone tell me how he makes his /r/ sound based on the sample?
/are used to indicate phonemes (speech sounds). Actually, your question asks what's "up with their "r" sounds". My suggestion is that the question is wrong, there is something 'up with' your non-native perception.