There are several considerations; in some cases they might have acted jointly, and/or sequentially.
Preface: syllabaries
As far as I can tell, the default structure for a script is a syllabary (where each sign represents a syllable), such as Cherokee, Linear B, or Japanese kana.
When scripts are invented by people not previously familiar with other scripts, or when they are simplified from a previously more complicated script, the result is usually a syllabary rather than an abjad (a consonantal script) or an abugida (a mainly-consonantal script with vowel diacritics).
So why are there so many abjads and abugidas? And where do alphabets come in?
Consideration 1: Afro-Asiatic grammar
As mentioned in the answer by Draconis, Afro-Asiatic languages have a grammatical system where word roots mainly consist of consonants, and grammatical inflections (as well as some forms of word derivation) are mainly indicated by vowels between those consonants.
In such a system, it makes sense to only indicate consonants in the script, to point out the root instead of focusing on the specific grammatical form.
Afro-Asiatic languages include Ancient Egyptian, as well as Semitic languages, including Phoenician and Ugaritic. All three of those were early adopters of consonantal scripts; the last two possibly borrowed the concept from the first.
(Akkadian was also a Semitic language, but Sumerian was not, and Akkadian script was borrowed from Sumerian. Akkadian cuneiform is not consonantal, but logosyllabic, somewhat similar to Japanese script in structure; some of its descendants, such as Hittite cuneiform, are almost purely syllabic.)
Note that Egyptian hieroglyphs were not a direct consonantal script; each hieroglyph typically represented a particular word, and by extension the consonantal structure of that word, which could be shared with different words.
Later some hieroglyphs evolved to only represent some consonants of a word, such that eventually there was at least one way to write every consonant by itself (allowing writing down arbitrary sequences of consonants, such as foreign names), but multi-consonantal symbols (which represented many consecutive consonants) persisted throughout the script's history.
There were also determinatives, signs that related to the meaning of the word being represented rather than its sound. (An approximate English comparison is the use of capital letters for proper names.) Egyptian was a complicated system, and the hieroglyphs in particular (there were simpler shorthand forms) actually became more complicated in later periods.
In turn many other scripts are ultimately descended from the Phoenician script and/or its close relatives (a scantily attested script close to the common ancestor of those relatives is traditionally designated as Proto-Sinaitic); some of them (notably Greek) had innovated vowel letters on the same level as consonant letters, and the script we use in English descends from one such branch.
Others did not do that, and remained purely consonantal, or had vowel diacritics rather than letters.
To the best of my knowledge, all non-modern alphabets, with one possible exception (Irish ogham), are either the result of vowel letter innovation in previously consonantal (or vowel-diacritical) scripts, or descended from such a result.
(There are, of course, many modern alphabets that are not directly descended from previous scripts, invented by people who were familiar with existing alphabets and assumed that an alphabet was the default writing system. It is possible that Irish ogham was an ancient case of the same idea.)
Consideration 2: default vowels
So, now we have a popular consonantal script (Proto-Sinaitic/Phoenician), and many languages, including many non-Afro-Asiatic languages, are borrowing the script (or one of its descendants) for their own use.
In non-Afro-Asiatic languages, roots are (usually) not made out of consonants, and grammatical inflections feature both consonants and vowels. So a purely consonantal script is inconvenient; some representation of vowels is useful.
Some scripts, such as Greek, Yiddish, and Hangul, used separate vowel letters; others did not. What could have influenced the choice?
One likely consideration is default vowels.
In many languages, a single vowel, usually /a/, accounts for the majority of vowels in text. In such languages, it makes sense for a consonantal character, such as /b/, to represent the syllable /ba/ rather than just the consonant /b/; then the comparatively rare cases where the consonant is followed by a different vowel can be represented by special signs for that vowel.
(Such a setup is usually referred to as an inherent vowel; it requires special handling for consonant clusters, because just writing the consonants next to each other would represent separate syllables.)
This was the option chosen in Brahmi and Kharosthi, both scripts for Indo-Iranian languages with a strong default vowel tendency, and the (many) Indic scripts descended from those two consequently also frequently use this setup.
Note that the "special signs for that vowel" do not have to be diacritics; there are some scripts (both within and outside the Indic family) that use this setup but are otherwise alphabetic, with full vowel signs.
Side-note: Afro-Asiatic vowel representation
As previously noted, Afro-Asiatic languages tend to have grammatical systems that make vowel representation less obviously required than for most other languages.
However, even in those cases, having some way to represent vowels was useful (especially for liturgical texts, where an exact reading was considered to be important).
As such, vowels had to be represented, but since the consonants were more important (featuring in roots rather than merely in inflections), the vowels could be relegated to diacritics.
This had happened independently in Hebrew (niqqud), Arabic (harakat), Ge'ez (where the vowel diacritics fused with the consonants, resulting in a Brahmic-like syllabary), and several other Semitic scripts.
A common alternate option was to use the letters for consonants that had grammatical correspondences to vowels, and/or became vowels by sound changes; this especially involved the so-called "semivowels" /j/ and /w/, which corresponded to vowels /i/ and /u/.
This option is known as matres lectionis [Latin for "mothers of reading"], and is a major stepping stone towards an alphabet, especially when borrowed into a non-Afro-Asiatic language; in fact it had been suggested that the Greek alphabet is based on a variant of Phoenician that had already made much use of such a system.
Other cases of this kind of system developing into a full alphabet include Yiddish, Sorani, and Mandaic (a Semitic language).
Consideration 3: there just aren't that many vowels
It might surprise speakers of Germanic languages (which are known for particularly large inventories of phonemic vowels) that most of the world's languages have just five or six distinct vowel phonemes (and many have even less).
This lets writing systems get away with using diacritical signs to represent vowels, because there are few enough vowels for visually distinguishable diacritical signs.
Note that the languages that the alphabets were invented for typically also had single-digit numbers of vowel phonemes, and consequently single-digit numbers of vowel letters.
This meant that languages with double-digit vowel inventories had to get... creative... with representing their vowels right. (Common solutions were digraphs/trigraphs, as in English and French, and diacritics, as in Hungarian and Vietnamese.)
But it also meant that abugidas, where vowels were diacritics in the first place, could hardly deal with large vowel inventories at all. This is part of the perceived dominance of alphabets: they're a lot more convenient when your language has a lot of vowels!
However, Afro-Asiatic (especially Semitic) languages don't usually have all that many vowels (and many of the vowels they do have are often associated with specific consonants and written as such), so abugidas do not get as much pressure from alphabets.
In theory, languages with small vowel inventories could use an abugida even if they did not historically borrow one from elsewhere; in practice, this almost never happens outside of the Arabic and Indic zones of influence, because most of the world's major languages use alphabets, so inventors usually default to an alphabet. One example of an invented abugida is the Mwangwego script, for the Chichewa language of Malawi (5 vowels).
As it happens, Australian Aboriginal languages tend to have a lot of consonants but very few vowels (usually just three), making them nearly ideal candidates for an abugida. I wonder if any Aboriginal inventors had came up with an abugida script by now...
Postscript: the other side
As mentioned in the question (and in the answers), there are many writing systems where the consonants are full letters, but the vowels are merely diacritics.
However, in principle, there could also be a script with full vowels and diacritic consonants.
And it turns out that there is in fact just such a script! In the Pahawh Hmong script of northern Laos (said to have been invented by an illiterate Hmong man in the 20th century), the vowels are full letters, but the syllable-final consonants are always diacritics, and so are some of the syllable-initial consonants (it's complicated).
