The reason is simple - Japanese and Chinese read kanji differently. 露西亜 is read roshiya in Japanese and is a good transliteration for English "Russia", but 露西亞 is read luh-hsi-ya in Mandarin and sounds nothing like "Russia". Well, neither does 俄羅斯 sound like "Russia" in either English or Russian, but this translation came from Mongolian, not directly from Russian. This is why Russia is 俄 in Chinese but 露 in Japanese.
When Mandarin wasn't predominant in most of China, many regions of China did have their own translations of country names based on their local dialects. For example, Russia was called 羅宋 in Shanghai, where translation is based on the local Wu dialect. The name 羅宋 is still used in China now, as the name of the adapted version of the Russian soup borscht, 羅宋湯 (literally "Russian soup").
The same applies to most names. Sometimes Chinese and Japanese do use the same translation (e.g. Britain and Colombia), but this is purely coincidental.