I guess most signed languages are known to too few speaking people for
such an occurrence to be likely. It is not so much a matter of prestige as a matter of population size and communication.
However, there are also very limited
signed languages used by speaking people when they have to communicate
silently or from a distance. It would have a better chance for "word
borrowing" from the signed to the spoken language.
Thinking of that, I did think of one expression in English that seems
to answer the question: "to give the finger" (sorry if that example
may break some rules - this is linguistics here). There are probably
some others examples. Maybe, "to wave someone goodbye" would qualify too.
The interesting point about "to give the finger" is that it can be used in contexts where it is not actually describing a physical movement, but can be interpreted directly as meaning strong refusal.
A similar analysis applies to "waving goodbye", meaning "aknowledging
departure or loss". It can sometimes refer a description of the
actual signed gesture, like "giving the finger". But it clearly only have the abstract acknowledgement
meaning when applied to something other than a person. For example, in
the sentence "if you trust bankers, you can wave your savings
There are also counter-examples.
The expression "Back-scratching" refers to exchange of favor, not to
actual physical scratching. However, it is irrelevant here because
scratching someone's back is not considered a mean of communication.
The expression "to nod approval" does refer to the signed gesture of
approval, but (afaik) is only used to actually describe the signed
gesture. One would not say "the man nodded" to mean that he expressed
agreement by any mean, but only to mean that he expressed agreement by
actually nodding. So the verb "to nod" cannot be considered as
borrowed from the signed language, but only as a reference to it.