Most South-West languages of Slavic language family, like Serbian, Bosnian and Montenegrin, include the Latin letter in their alphabets, which has not been a part of Cyrillic writing system they're originating. How the letter find it's way to modern Slavic Cyrillic languages, despite having sound-alike letter <я> in the earliest recorded Cyrillic?
The letter <j> is really used in some Cyrillic-based alphabets, all of them were once created either by a certain person or by a group of people, that is, these alphabets aren't a product of natural evolution of script.
The ones you're interested in are the oldest ones, the Serbian and Macedonian alphabets, Macedonian is actually an adaptation of the Serbian alphabet to the Macedonian language.
Karadžić reformed the Serbian literary language and standardised the Serbian Cyrillic alphabet by following strict phonemic principles on the Johann Christoph Adelung' model and Jan Hus' Czech alphabet. Karadžić's reforms of the Serbian literary language modernized it and distanced it from Serbian and Russian Church Slavonic, instead bringing it closer to common folk speech, specifically, to the dialect of Eastern Herzegovina which he spoke.
Karadžić used the principle "one letter = one phoneme", so he needed a letter for /j/ which is a distinct phoneme in the Slavic languages. Also, he wanted to distance the new alphabet from the Church Slavonic alphabet that uses the letter <й> for [j] (Russian and Bulgarian Cyrillic also use <й>), so he chose <j> as an alternative for <й>. Czech, Polish, Croatian, Slovene, German, Hungarian — all the major languages in those parts — use <j> for [j], so his choice is clear.
The Macedonian Cyrillic alphabet was standardized in 1945 by a committee formed in Yugoslav Macedonia after the Communist Partisans took power at the end of World War II. The alphabet used the same phonemic principles employed by Vuk Karadžić, hence the use of <j>. Note that the Macedonian Cyrillic alphabet also uses the letter <s> which may seem to be derived from the Latin alphabet, but that's not so, <s> is a part of the original 9th-century Cyrillic alphabet and it stands for [d͡z]
Around the edge of the 19th and the 20th centuries, the so-called Drahomanivka, one of the Cyrillic-based orthographies for Ukrainian, an East Slavic language, also used <j> for [j], and also under the influence of Vuk Karadžić. Even now, on the 20 hryvnas Ukrainian banknote one can see verses by Ivan Franko with that <j> in the Ukrainian Cyrillic text, Franko lived in the Austro-Hungarian Empire and during his lifetime Ukrainian didn't have a standard orthography.
Besides, by the 1940s the Soviets had created Cyrillic alphabets for most of the languages of the Soviet Union, dozens of them, many with complicated phonology, so Latin letters were used in some of them. For example, our letter <j> was a part of the official Cyrillic alphabets for Altai, Azerbaijani, Kildin Sami, Orok, and Ossetian. Other Cyrillic alphabets also use <w> and <q>.
Each letter in each alphabet has history and a fascinating story behind it, it's easy to learn those things, start with searching Wikipedia just for a single letter, most of them have a separate article, even <j>.
The Latin letter "j" is a post-classical development from the letter "i", having originally been just a variant of "i", and came to stand for a distinct sound when applied to German. The phonetic value employed in Romance languages (and English, via French) was different. So the use of "j" in West Slavic derives from analogous usage in German.
The same place the Latin letter <j> comes from in other languages: Latin
In Latin, the letter <i> could represent both a vowel [i]/[i:], or a consonant [j] (if preceding another vowel). It also had a longer and a shorter written variant that, to start with, seem to have been in free variation
As time went on, the consonantal pronunciation generally shifted to [ʒ] (and later to [x] in Spanish) when it occurred at the beginning of a word or between vowels meaning the two sounds no longer formed an obvious pair. This helped encourage the two written variants to split so that the short one became modern <i> for the vowel, and the long one became modern <j> for the consonant (a similar thing happened with <u> and <v>). Later, the largely obsolete letter <y> (which was originally only used in Greek loans which had an upsilon) came to be used for remaining (or newly borrowed) instances of [j]
Thing is, the shift of [j] to [ʒ] only really happened in the Romance languages. The Germanic (apart from the English) and (Catholic) Slavic peoples had no reason to change how they pronounced <j> and so carried on pronouncing it the same as they always had: [j]
The letter <ј> is present in some Cyrillic alphabets though, borrowed from its use in various Latin alphabets for Slavic languages. It can be found in both Serbian and Macedonian Cyrillic (as well as in several non-Slavic Cyrillic alphabets)