What I am interested in knowing is what exactly a specific diacritic does to a sound, and which muscles in the vocal tract are responsible for making those sounds.
A good and official source for IPA letters is here. From it you can learn exactly what each diacritic is called, e.g. ̥ is "voiceless". This also includes illustrative performances of most of the IPA letters, although there is a gap for the vowel-nudging diagritics. To go with that, I suggest Ladefoged & Maddieson The sounds of the world's languages, where you can look up particular terms in the index and see what they say about it. There is no unified resource which spells out the acoustics and articulation of each diacritic. In general, there is very little data out there which documents sound differences at the muscular level. IPA diacritics are particularly hard to define, because they refer to a modification of something more basic. The exact acoustics and articulation of palatalization depends on what sound you are comparing with, so it's different to some extent in [pʲ, tʲ, qʲ].