I can't pretend to really understand this article, but the authors have an algorithm for encoding speech at 800 bps (as they say, using split vector quantization). They have tested it using male and female voices and seem satisfied that the output is comparable to standard 2400 bps LPC-10 encoding, though the authors are electrical engineers, not psychologists, so there isn't a body of perceptual studies to go with the method. On the other hand, this study does have a perceptual study to go with the technology, and an algorithmic tweak (based on a theory of auditory perception) results in improves perceptual quality for 2400 bps LPC coding. Chapter 2 of the book Ultra Low Bit-Rate Speech Coding seems to be out there (here). It appears (and I must emphasize "appears") that one can reach 120 bps coding. It might be worth posting a related question to Signal Processing SE, to see especially to what extent these ridiculously low rates yield intelligible speech (esp. to get a technically-qualified evaluation of the state of the art).
This thesis suggests 20 bits/sec, based on Shannon's estimate of 1 bit per character entropy in English text, 20 phonemes/second speech rate, and about 1 character/phoneme.