Jlawler's comment contains the direct answer to the question. Definite personal pronouns (I/me, you, he/him, she/her, it, we/us, they/them) cannot take a restrictive modifier. In other words, they cannot take a dependent that narrows the set of entities that they denote. This trait of personal pronouns underlies their use as test words for constituent structure. For example:
(a) The man with the hat knows the woman with the scarf.
(b) He knows her.
(c) *He with the hat knows her with the scarf.
Sentence (a) is the starting sentence. Sentence (b) shows proform substitution; the personal pronouns he and her have been substituted in for the noun phrases the man with the hat and the woman with the scarf. Based on the acceptability of sentence (b), one concludes that both the man with the hat and the woman with the scarf are constituents. Definite pronouns such as he and her (and them) take the place of constituents, in this case of complete noun phrases.
The unacceptability of sentence (c) reveals that the strings the man and the woman in (a) are not constituents. In other words, the definite personal pronouns he and her cannot take dependents (=modifiers), since they necessarily replace an entire noun phrase. This fact explains why them who lost their money in the question is bad English. The relative clause who lost money is a postdependent (=postmodifier), and as such it cannot modify them (because them as a definite personal pronoun cannot be modified).
The plural demonstrative pronouns (these and those) behave differently. They can take postdepndents (=postmodifiers, i.e. a modifier that follows them), e.g
(d) These with hats know those with scarves.
This is simply a trait of the plural demonstrative pronouns (these and those) -- there is no good explanation why plural demonstrative pronouns behave differently than definite personal pronouns; they simply do. Note that the plural demonstrative pronouns also behave differently than the singular demonstrative pronouns in this regard, e.g.
(e) *This with a hat knows that with a scarf.
Singular demonstrative pronouns (this and that) are behaving like the definite personal pronouns; they cannot take dependents.
The combination plural demonstrative pronoun + restrictive relative clause can actually be viewed as a particular construction in English and related languages. That is, it is a combination that occurs relatively frequently and has therefore been lexicalized. German has a very similar construction, e.g.
(f) Diejenigen mit einem Hut kennen diejenigen mit einem Schal.
those with a hat know those with a scarf.
By acknowledging that one has a particular construction, one is in a sense admitting that there is no real grammatical "explanation" for the phenomenon. It simply exists.
Finally, note that there are certain apparent exceptions to the principles mentioned above. There are uses of personal pronouns that actually allow modification, e.g.
(g) He who studies a lot gets a good grade.
In this example, the personal pronoun he is not referring directly to a specific entity, which means it is not definite; it is, rather, being used as an indefinite pronoun; it means 'the one, anyone', e.g. Anyone who studies a lot gets a good grade.