Are there languages in which lexical tone can associate to semivowels or glottal stops, or does tone ALWAYS associate only to vowels when it is realized in a spoken word?
"Associating" is a phonological relation, pertaining to the distribution of tones relative to segments. The notion that a segment is capable of having the association relation with a tone is abbreviated by saying that such and such segment is a Tone Bearing Unit (TBU). Being a phonological relationship, there can not only be a considerable degree of variation between language in terms of the segments that define TBUs, there may also be derivational variation within a language (some segments only become TBUs later in the derivation). There is a point at the end of the phonology when things become murky and we stop talking about tones associating, instead talking about "realization of tone as F0" (i.e. we look at pitch tracks). I will forego an extensive discussion of relating the latter kind of continuous phonetic event to categorial segment / tone relations.
The most basic generalization about TBUs is that vowels are always TBUs, thus are capable of (directly or indirectly) bearing lexically-specified tones. In saying "directly or indirectly", I'm referring to an extended version of the simple dominance-type relation that is the formalization of "associate" in autosegmental phonology. The underlying issue, with reference to an string [bá], is whether the tonal object is immediately dominated by the laryngeal node of the vowel, the segmental root node, its mora, or the syllable containing that vowel. Moreover, since tones are not unstructured letters (they are structures defined by combinations of features), we have to settle on what thing we mean by "a tone". Conventionally, we say that a tone is the totality of features defining a tone, more verbosely called a "tone root node". While I am convinced that the best solution is that tone root nodes (i.e. tones) are immediately dominated by moras, there have been many proposals about what exact thing tones associate to. So tones at least indirectly associate to vowels in that they are immediately dominated by moras, which immediately dominate segmental root nodes.
Then the question is, what kinds of segments can be moraic? Just asking that much, the answer is, "any". However, it is well-known that there are major asymmetries between vowels and other segments, in terms of their ability to be moraic. Dissertations and other research by Draga Zec and Bruce Morén-(Duolljá) have explored this topic (though non-tonally), and there are some typological generalizations about what segments are optionally moraic (optional in the sense "may be, but are not always in all languages and items"). Moraic consonants generally are found in the syllable coda, although see work by Jen Muller and Nina Topintzi on non-coda geminates (moras being implicated via the credo that geminates are moraic). One language that figures prominently in the discussion of geminate consonants and moras is Luganda: all non-continuants can be geminate, and they act as TBUs.
The most common non-vowel which is tone-bearing is the syllabic nasal. An example is Logoori [ḿꜜkwáasi] 'in-law'. This is probably due to the fact that nasals are most susceptible to being autonomously syllable. Looking at post-vocalic coda segments, the main generalization is that sonorants (liquids and nasals) are most capable of being tone bearing. Glides will be kicked down the road for a while. Although [h, ʔ] are sonorants under SPE criteria, they do not appear to enter into the "asymmetrical coda consonant" class. When we get to the point of asking whether obstruents can be TBUs, we are now in the realm of rare events: they may be, but are usually not. Now turning back to laryngeal glides, there are languages (Chinese languages especially) where glottal stops are TBUs, but I am not aware of any language where glottals and not other obstruents are TBUs. It may be that the basic TBU cut is vowels / sonorants / consonants.
As for the status of glides as TBUs, there is a problem of circularity of analysis here: glides almost never minimally contrast with glides. If one encounters something that sounds like it has the segments "maw" and has a falling tone (when contour tones require two TBUs), we would usually say that the string is [máù], and if there is only one tone, we would say it is [máw]. Sometimes, you encounter transcriptions like [máẁ] (e.g. in Akan), but the transcription there is being led by a phonological premise, that there are no vowel sequences in the language. So it is extremely difficult to determine whether you are dealing with a tone-bearing glide, or a tone-bearing vowel in a monosyllabic (or even disyllabic) vowel sequence. Morton in her dissertation on Anii discusses the interesting problem that there is a phonological contrast between vowel-vowel sequences and vowel-glide sequences. In this language, vowel-vowel sequences "act short" (true diphthongs have only one TBU), but sequences of V-nasal, V-glide and long vowels "act long" (have two TBUs).
Finally, briefly addressing the phonetic issue, people also use "associate" in a phonetic sense to refer to whether a given segment has a surface-distinct pitch pattern, that is, the consonants "are pronounced with their own tone". This matter really can only be addressed in the context of a full phonetic analysis of F0 ~ tone realizations in a language, without which you could pretty automatically reject potential examples of surface tone-bearing consonants by saying "That's just X" (for a small range of Xs). Examples like Logoori [ḿꜜkwáasi] are hard to trivialize, whereas [ḿsáára] 'tree' can be eliminated from the set of hard-core examples by saying that it is really [m̩sáára] with higher pitch phonetically anticipated on the preconsonantal nasal. Bearing all of that in mind, at the phonetic level (and by definition), any segment "with phonetic tone" must be voiced. As example of a surface-distinctive tone bearing (voiced) obstruent in Logoori is [d́ꜜdíiji] "wall" (compare [ddɪ́kʊ] "day"). Such examples are extremely rare, and not well-studied from a phonetic perspective in any language (the main other candidate that I know of being Luganda). My actually final point will be that Nantongese Chinese (and possible other dialects) have some unusual vowels, which sound like syllabic voiced fricatives, and are fully tone-bearing throughout the phonology. This is conceptually related to Mandarin "syllabic z". In Mandarin, these only appear after sibilants and may be reduced to being an allophone of some other high vowel; this does not appear to be possible for Nantongese. However, from the perspective of phonological function, these things are vowels, just not the usual unconstricted kind of vowel.