The problem with the language/dialect distinction is that it always, always, always involves politics. Even when trying to do things on a purely linguistic basis, there are far too many borderline cases and questionable situations (e.g. dialect continua, one-sided intelligibility, etc) - you will never end up with a wholly unbiased classification.
That said, most (maybe even all) non-Japanese linguists recognise Ryuukyuuan as a group of languages separate from Japanese, regardless of how many Ryuukyuuan languages they count. I would guess that the 'canonical' Japanese view, that Ryuukyuuan is just a set of Japanese dialects, comes from nationalism and the very inaccurate and outdated view of Japan as an almost totally homogenous country - even Ainu was called a 'dialect', and it's totally unrelated to Japanese! I don't know how many Japanese linguists still regard them as dialects instead of languages, though.
The linguistic criteria fairly unambiguously support the separate-language analysis - there is a clear gap in intelligibility between the Kyuushuu mainland and the Amami archipelago. It actually turns out that Ryuukyuuan as a whole is genetically closer to Northeast Kyuushuu than anywhere in the south, but it took until 2003 (in a paper by Leon Serafim) for anyone to notice. If you show Japanese speakers a video of someone speaking Ryuukyuuan (without explaining it) and ask them if it's Japanese, they will definitely say it's not - I actually came across a video on YouTube recently that was a quiz asking the viewer to distinguish audio clips of Japanese dialects and Ryuukyuuan languages from clips other East Asian languages :P
Now, it may turn out that Ryuukyuuan is genetically a sub-branch of a Japanese-internal dialect group, but it has been innovative enough since it split off that it still counts as a group of separate languages.
I don't know about Hachijou - I know it's very very different from the rest of Japanese, but I don't know if there's a consensus as to a separate status. If I were to venture a guess, I would bet that not only Hachijou but also a number of other dialects from Japan's main islands ought to be considered separate languages rather than dialects - though these might be a bit more muddled by the dialect continuum problem than Ryuukyuuan or Hachijou.