WARNING: The question is sooo many-sided, it is very wide and can be split into at least 3 different questions. I'll answer it all, don't tell me later that you haven't been warned the answer would be long.
First of all, this letter has no sound of its own. The main function of the soft sign <Ь> in Russian is to change the sound of the consonant letter which stands before it. In Russian, most consonant sounds can be paired into couples of non-palatalized ("hard") vs. palatalized ("soft"). In English no consonant sound has such a pair; most of them are non-palatalized and for English native speakers it is usually a hard part of studying Russian pronunciation to learn how to pronounce those sounds palatalized.
Palatalization (P) can easily be explained, but is very hard to produce with the tongue (for those who have no such thing as P in their native language). The main idea behind it is that palatalized consonants (PC) are bi-focal: beside the main focus of articulation, the tongue also makes the second focus touching the palate with its back, hence the name. For example, Russian <C> is approximately like the English [s], with the tip of the tongue nearly touching somewhere near the upper teeth, but in the Russian <CЬ> [sʲ] the tongue also touches the palate with its back adding a [j]-like air to [s]. Still, [sʲ] is one sound, you can drawl, sustain it for a long time and all that time it will sound the same. (Not all palatalized consonants can be sustained, of course, just as not all consonants can be sustained.)
Another trick is that the P of a consonant sound can be shown not only with the soft sign, but also with vowel letters. They are also paired into the ones that cause P (they are called "iotizing" — that is, 'adding the [j] air') and the ones that don't (called "non-iotizing"). The pairs are like this:
Non-iotizing vowel letters: a э ы о у
| | | | |
Iotizing vowel letters: я е и ё ю
The idea behind the iotizing vowels is that when they stand after a consonant letter they palatalize it and then they are read as the corresponding non-iotizing letters. For example:
мал [mal] - small
мял [mʲal] - (he) crumpled
In the beginning of words and after another vowel letter the iotizing letters are pronounced as [j] + the corresponding non-iotizing letter:
ясно [jasnə] - clear
бояться [bə'jatsə] - to be afraid (yeah, here the soft sign doesn't affect any sound)
But what if we need to write a word in which a consonant is followed by a distinct sound [j] + a vowel, like a 3-sound word [bʲju] - 'I beat'? We cannot write it as
бьйу, because (another trick is coming up!)
the combination <Й> + a non-iotizing vowel letter are (hahaha!)
forbidden in spelling of Russian words which were not borrowed from another language. 
If we write it as
бю, then according to  it will be read as [bʲu], a 2-sound word... The only correct way to write that word is
after <Ь> the iotized vowel letters are read as [j] + the corresponding
шью [ʂju] - I sew
пью [pʲju] - I drink
And at last, differentiating the gender of nouns with the help of <Ь>. In Russian, most feminine nouns end in [a], but there are lots of them that end in a consonant. In the latter case a soft sign must be written at the end of the word (ahaha!) irrespective of whether the final consonant is actually palatalized or not!
(masc.) мел [mʲel] - chalk
(fem.) мель [mʲelʲ] - shoal
(fem.) мышь [mɨʂ] - mouse (ends in a non-P consonant!)
(fem.) ложь [loʐ] - a lie (ends in a non-P consonant!)
This is actually everything you asked about! Naturally, there are other issues with <Ь>, like differentiating some verb forms where it doesn't in any way affect the pronunciation, or using it before non-iotizing letters in borrowed words, not to mention its historical usage problems, but that is another tale.