Latin "b" apparently developed a pronunciation variant [β] (a voiced bilabial fricative or approximant) at some point that was used between vowels. Presumably there was a period when the shift from [b] to [β] in this context was incomplete. One piece of evidence for this shift is spelling confusion between "u" and "b" in some Latin texts, which is attested as far back as the first century AD.
The article "Lenition and phonemic contrast in Majorcan Catalan", by José Ignacio Hualde, Marianna Nadeu & Miquel Simonet (in Romance Linguistics 2009, edited by Sonia Colina, Antxon Olarrea and Ana Maria Carvalho), says that
Whereas we cannot be sure when /b/ started to admit approximant realizations or even when these approximant realiations of intervocalic /b/ became the usual articulatory target, we know that /b/ merged with /w/ by the first century of our era. This is because around this time the graphemes B and V start getting confused (Allen 1978: 41). We also know that all Romance languages have merged the results of Classical Latin intervocalic -V- and -B- phonemes. The details of this evolution appear to have been the following. First, /b/ acquired approximant allophones in intervocalic position: [b] > [β]/V_V, as in HABERE [abe:re] > [aβe:re]. At this point, we would still have a contrast between /b/ and /w/, since HABERE [aβe:re] would contrast with LAVARE [lawa:re]. Then the contrast was lost, through another change affecting [w], which became labiodental, [w] > [v], as in [lawa:re] > [lava:re], thus reducing the distance between the two phonemes. A subsequent process was the merger between [β] and [v], which is found in all Romance languages
I'm not sure what evidence the detailed description that Hualde, Nadeu & Simonet give of the stages of the sound change is based on. Maybe someone else will be able to say more.
Some other SE posts with relevant information:
On Linguistics SE:
How did the letter “v” come to represent the voiced labio dental fricative?
A comment on the answer by Alex B. mentions "the w ~ β merger in Vulgar Latin, which included w > β and simultaneous b > β (intervocalic position only) (Weiss 2009/2011, p. 512)." This description seems to conflict with Hualde, Nadeu & Simonet's account of the development of Classical Latin [w] > [v].
On Latin SE:
Welsh had its own intervocalic lenition processes, actually, so I don't know if the form of the Welsh word llyfr tells us much about the pronunciation of the Latin source word.
A nitpick on the side: As far as I know, the pronunciation of Welsh "llyfr" is not represented accurately by "liffer", so I'm not sure why you suggest it is pronounced like that. The initial consonant is supposed to be a voiceless fricative [ɬ], and the middle consonant is supposed to be voiced [v]. In Welsh orthography, the single letter "f" represents the sound [v], while the voiceless fricative [f] is represented by the digraph "ff". The traditional way of approximating the pronunciation of "llyfr" using English sounds would be something like "thlivver".
The reconstructed Latin pronunciation of "liber" meaning "book" could be written as [ˈliber], [ˈlɪbɛr] if we want to get more specific about vowel quality, and [ˈlɪβɛr] if we want to represent a pronunciation with lenition of intervocalic b. This is something like English "libber" or "liver" (the organ), if you want an approximate comparison.