Both "poor" /pʊər/ and "doom" /duːm/ represent regular developments of pre-GVS /oː/ in modern standard English. The difference is in fact the context. The vowel /ʊə/ is the normal reflex before an /r/, and the vowel /uː/ is the normal reflex in other contexts (in fact, when dealing with a dialect that still retains syllable-final /r/, it's possible to analyze [ʊə] as simply an allophone of /u/).
However, /ʊər/ has historically been unstable as a phoneme; there are a wide range of possible realizations, and for many speakers in many dialects it has merged with other phonemes. The tricky part is that which phoneme it merges with depends on the dialect and on the particular word. It may merge with /ɔːr/ as in "more", or disyllabic /u.ɚ/ as in "doer", or (especially after /j/) with /ɝ/ as in "nurse." For more details, you can look at the Wikipedia article on "English-language vowel changes before historic /r/".
So the only issue is with the pronunciation of the word "door." According to the Online Etymology Dictionary, "door" actually appears to continue two OE words:
Middle English merger of Old English dor (neuter; plural doru) "large
door, gate," and Old English duru (fem., plural dura) "door, gate,
wicket;" [...]
Middle English had both dure and dor; form dore predominated by 16c.,
but was supplanted by door.
It's not totally clear to me from this what the pronunciation of the word was during the Middle English period.
The Oxford English Dictionary discusses this in their entry on "door," and also seems unsure of the development of the modern pronunciation:
The spelling door points to an earlier pronunciation with ū or ū from
Middle English close ō , which is further attested by Scots dure /dør/
(also in Cath. Angl. 1483), and is considered by Luick as a northern
lengthening of Old English u . The current pronunciation may be a
retention of that evidenced for 16th cent. dore by quot. 1597 at sense
1aβ. ; but it may also be a more recent modification of /dʊə(r)/ , as
in the case of floor, and vulgar pronunciations of moor, poor, as
more, pore.
[...]
1597 Shakespeare Richard II v. iii. 75 Open the doore, A beggar begs that neuer begd before.