So far, this is a short general post about some exceptions in which palatalization of /g/ before a front vowel did not happen.
This is a well-known phenomenon and virtually all books on historical phonology of English worth reading devote a special section to this topic - see e.g. Hogg 1992, Lass and Anderson 1975 or Minkova 2014. cf. Durkin 2014: 195
See the quote below from Lass and Anderson 1975:
It is even discussed in slightly "less technical" literature, e.g.
Upward and Davidson 2011 write the following:
One recurring feature in words of Scandinavian origin that sets them apart from Anglo-Saxon-derived words is the retention of /g/ before a front vowel, as in get, give, begin (if these words had derived from Anglo-Saxon English, it should have resulted in Modern English forms such as * yet, * yive, * beyin)"
(pp. 28-29; emphasis mine - Alex B.).
Sara Pons-Sanz (2015) traces this idea back to Björkman 1900.
As for Scandinavian influence on English vocabulary, I strongly recommend reading at least Part IV. Scandinavian Influence (pp. 171-221) in Durkin 2014, Borrowed Words (Philip Durkin is the chief etymologist of the OED, by the way).
One very important thing to remember - as Durkin reminds on p. 192
"in some of these instances, it is perhaps more likely that what we have is substitution of the Scandinavian form for its similar-sounding native cognate within a particular bilingual speech community, with the Scandinavian form then gradually spreading at the expense of the native form" [emphasis mine - Alex B.].
NB! by Scandinavian Durkin means "the ancestor varieties of both West Norse (Norwegian and Icelandic) and East Norse (Danish and Swedish), but at a time earlier than our earliest substantial surviving documents for any of the Scandinavian languages, and at a time when the differences between West and East Norse were still very slight" (p. 175).
Back to your OP - we don’t really know why, and this happens not infrequently in historical linguistics. For example, we don’t really know why OE ceorfan > PDE carve (and not *cherve).
As for OE beginnan, two explanations are possible.
1. Scandinavian influence (Dieter Kastovsky, e.g. Kastovsky 1992 specifically mentions OE beginnan):
However, there is no direct evidence to support this hypothesis but we may have some indirect evidence.
First, even though we can find OE beginnan, another verb was much more common – OE onginnan, until it was superseded by ME beginnen around the fourteenth century.
An interesting similar case could be the development of OE ongean (> PDE again). The ODEE (ed. Onions, 1966) says “the native forms in aȝ-, ay- did not survive beyond XVI, being superseded universally by forms in ag-, derived from Scand. and appearing first in northern and eastern texts XIII.”
More information about the chronology of this change could come from the development of OE giefan (> PDE give). The ODEE mentions that the non-patalal variant appears c.1200 and that the regular ME forms yive, yeve etc. “prevailed in southern and midland writings till XV.”
So, the remark by darkgamma that "the etymology is off and the chronology is wrong by a few hundred years" is irrelevant.
However, the biggest problem is the apparent lack of a Scandinavian (Old Norse) cognate, and I'm by no means the first linguist to notice this.
2. (Analogical) leveling:
However, there is no conclusive evidence to support this, either. We have to remember that even though it is “inherently irregular,” it is a “fairly systematic process” (Hock and Joseph 2009: 152; emphasis mine - Alex B.), which means we should be able to find other examples – very few but they should be out there.
Cf. Hogg 1992 “with the degree of allomorphic variation noted in §7.41, it would be expected that there should be considerable analogical levelling and extension” (Hogg 1992/2011: 269) and later he claims that “[a]mongst strong verbs there appear not to be clear-cut examples [of analogical levelling – Alex B.] in OE” (p. 270; emphasis mine - Alex B.).
However, in our case this would contradict the directionality of Mańczak’s seventh tendency of analogy, which states that “the forms of the present more often bring about the reshaping of other tenses than vice versa” (as cited in Campbell 2013: 265).
Examples of levelling (strong verbs, West Saxon):
OE gyllan – geal – gullon, cf. PDE yell (yelled)
OE gyldan – geald – guldon, cf. PDE yield (yielded)
Also, the analogical levelling hypthesis cannot explain why OE onginnan > ME aginnen was superseded by ME beginnen.
To conclude, as Pons-Sanz 2015 writes,
"In the end, it is up to the linguist’s judgement and his / her willingness to accept the Norse derivation of a term without a formal Norse imprint that determines whether or not it will make it to the list of medieval English words with a Norse etymon. This, of course, is a serious problem for historical linguistics, given the central place of many Norse-derived terms in the English language."