Historical Reasons
The most of these differences in English seem to be due to historical reasons.
"China" seems to refer to either the Qin province or the Qin Dynasty which comes to English via Persian.
Similarly, "Japan" seems to have come to English via the Portuguese name for Japan (itself a transliteration of nippon).
Finally, "Germany" refers to the geographic area called Germania in which Germany resides.
You can see a similar process at work on a much smaller scale with people who still refer to "Burma" (which changed its name to Myanmar), "Ivory Coast" (which has stated that its name is always "Cote d'Ivoire", regardless of the language), and "Czechoslavakia" (a country that no longer exists and has been replaced by both Czech Republic and Slovakia, officially Slovak Republic).
Pronunciation
Other times these differences can, as you noted, occur for simple pronunciation reasons. This is why we refer to "Spain" instead of España. Also, as noted, the Japan example from above may fit this category as well.
Adaptations in other languages
During my short stint studying Chinese I was rather jealous that America became 美國 (meiguo, literally "beautiful country") and England became 英國 (yingguo, literally "hero country") while my native Canada was simply 加拿大 (jianada), a mere transliteration.
However, the truth is a bit more complicated. Meiguo seems to be derived from a transliteration of "America" that got shortened and affixed with guo (country). Similarly, yingguo seems to also be derived from a shortened and affixed transliteration. So these country names are both a transliteration and an adaptation at the same time.
Interestingly, American and England seem to have come to Korean via Chinese (미국, miguk, and 영국, yeongguk, respectively) while France (in Mandarin 法國, faguo, "law country". Presumably via the same mechanism as America and England), which would be koreanized as 법국 (peopguk), is actually 프랑스 (phurangsu, a more faithful transliteration).
Some other interesting adaptations of country names in Korean are China and Japan. Korean, like many East-Asian languages, borrowed a lot from Classical Chinese (despite being linguistically unrelated). As such, many Chinese characters have a Korean pronunciation. This is how we can tell that 미국 and 영국 are derived from the Chinese. As you have stated, the Mandarin name for China is 中國 (zhongguo). Using the Korean pronunciations of those Chinese characters we get the Korean name for China: 중국 (jungguk). Similarly, Nippon is written in Chinese (technically, in Kanji) as 日本 which gives us the Korean name for Japan: 일본 (ilbon). Oddly, adaptations of this sort do not extend to city names but do sometimes apply to people's names.