By way of analogy, English speakers find it relatively easy to learn romance languages, or Germanic ones. Much Middle English vocabulary was introduced by the Norman conquest, and much Old English vocabulary was derived from Germanic influences via the Saxons and Jutes. Approximately 60% of Vietnamese vocabulary derives from common sources shared with Cantonese - particularly from the Bai Yue ethnolinguistic complex. Simultaneously, Mandarin derived from the confluence of Cantonese antecedent dialects and the Manchurian language. Common sources seem adequate to explain the bulk of the affinity.
In general, phylogenetic purity is a characteristic of didactic simplifications, rather than of adequate explanations. As in biology, so too in diachronic linguistics: The generative axioms of signification may derive, for example, from the major language group, while vocabulary is dominated by sources in geographic neighbors of another group entirely. While I am not expert in Vietnamese historical etymology, it is my impression that Vietnamese may be an exemplar of this paradigm.
Certainly it is true that the borders of the southern Chinese and the Vietnamese dynastic empires have fluctuated significantly over the centuries of the historical epoch. In this regard, a closer Euro-centric analogy might be the Ukranian-Russian pair.