Segment in its most general sense can be taken to mean something along the lines of "small, discretely identifiable chunk of speech", but it is used in all three of the ways you wondered about.
It is often used as a synonym for both phone and phoneme, by people subscribing to the concepts of phones and phonemes; these people may use it with a directly equivalent meaning to phone or possibly even phoneme, but the latter usage perhaps more often appears clarified as phonemic/contrastive segment.
It's also used as a way of avoiding those terms, either because your analysis is still in its early stages and you are not making any claims about the phonemic or otherwise status of any observed phones, or because, as @jlovegren suggested, you are trying to avoid committing to these concepts altogether.
However, there is a third concept that is important to the idea of a segment that I think is worth noting here. Segment on its own generally refers to a bit of speech within a linear sequence of other bits of speech, but the development of autosegmental theory (see pretty much anything by John Goldsmith) basicallly allowed for speech to be represented on a series of tiers, instead of just a linear sequence, with things on higher-level tiers connecting to different parts of the tiers below them. A simple example would would be to have a tier that has the tones, and then another tier that has the phonemes/phones, with association lines showing which tones apply to which phonemes/phones.
Autosegment comes from autonomous + segment, because segments were more autonomous once they were freed from purely linear representations (which would have all the tones and everything bundled together).
So, segment can be used to mean 'segment in a linear sequence', in opposition to autosegment, which means 'segment in a tiered, hierarchical representation'. Autosegments can therefore include tones, nasalization, and various other things that would not usually be considered stand-alone segments.
Similarly, beyond autosegmental phonology, segment can also be used to mean 'the segmental (e.g. phonemic) material of a word/utterance', in opposition to suprasegmental material, which would be the tones etc. that exist 'above' the segmental material. This provides one distinction between segments and phonemes, if you work on a language in which tone is phonemic; the segments would be the consonant and vowel phonemes, while the tone phonemes (tonemes) would be described as suprasegmental features, or perhaps autosegments if you are using that approach.
In my own experience, I most often encounter segment used in these last two ways, to make a distinction between either autosegments or suprasegmental features. But, segment is used in very diverse ways by different linguists.