The labelling of phonemes is fairly arbitrary: we say English has an
/d/ phoneme, for example, but it may not ever be realized as a true perfect IPA
[d]. Reconstructed languages often have phonemes not named after any IPA symbol, such as Proto-Indo-European's
/*ǵ/ (what would it even mean to put a high-tone mark on a velar plosive?) or Proto-Slavic's
/*ъ/ (not even a Latin character). Some reconstructions (such as Alice Kober's early analysis of Linear B) have even just given each phoneme a number, to avoid committing to any ideas of what it might have sounded like.
So the phoneme in Japanese is usually written
/ɯ/, whichever the individual doing the analysis prefers. Neither is wrong: you could just as well call it "vowel number three" and write it
/3/, and that wouldn't be wrong either. What's important is just that readers understand what phoneme you mean, and neither
/ɯ/ (nor even
/3/) is likely to be mistaken for any other phoneme.
Phonetically, the most consistent features of Japanese
/u/ is that it's high and non-front. It sometimes moves as far forward as central
[ɨ] after palatals, and might have the lips rounded, unrounded, or compressed (
[ɯ̟ᵝ]). The version used in the prestige dialect of Tokyo in careful, formal speech tends to be more like
[ɯ̟ᵝ], so we might well call the phoneme
/ɯ̟ᵝ/, but that's just too annoying to write over and over.