In general, for any known language sound there is some probability that a given language will have that sound, so that the clicks are very rare sounds and [a] is a very common sound. Theoretically, you could say that the probability is N that a language would have any labial fricative, or M that it will have [ɸ], if only we had a uniform list of phonemes of human languages. However, we don't have such a database. It is a coincidence that that particular set of older languages do not have labial fricatives. Proto-Bantu, Proto-Salishan and proto-Semitic also do not have labial fricatives (caveat about Bantu below), also Menominee and Didinga do not have labial fricatives.
Fricatives are less frequent in human languages, compared to stops, because it is harder to identify them (their acoustic cues are not as robust). Labial fricatives are not the most difficult to identify ([θ] might have the honor there), but they are much more difficult to identify compared to sibilants (their characteristic spectrum is more diffuse and at a too-low frequency). What we don't know is whether [x] as a phoneme is statistically more-often attested compared to [f], or the opposite. It does seem that [f] is more frequent than [ɸ]. The evolutionary logic of this is that if there were [f] at some point, it could most easily change to something else like [h] or just get deleted, because it is hard to detect.
One should also exclude allophonic variants and only look at fricatives as phonemes. In many Bantu (and other) languages, there is a phonetic "fricative" [β] which may more properly be classified as an approximant, which is the result of weakening /b/ in some context. [f] developed allophonically in Hebrew and other Semitic languages, likewise.
