I would like to know how the relation of corpus size to observed ngrams is(especially unigrams). Since the word frequency follows a Zipf distribution, i would expect some kind of logarithmic relation as well, but has anyone done research on this particular question?
Thanks to Robert for pointing me at this. Here is an approach to approximate the number of word types in a corpus. We assume that Zipf's law holds and the frequency of the second most common word type is 1/2 the frequency of the most common word type and so on.
Let N be the corpus size. Let maxfreq be the (relative) frequency of the most common word type. Then by Zipf's law, it should be
freq(w) = maxfreq / rank(w)
where rank tells us at what rank(w) the word type is with respect to its frequency, and freq(w) is the relative frequency of that word type.
The number of word types can be approximated by the maximal rank of words that have an absolute frequency higher than 1.
s.t. N*freq(w) >= 1
that means that this maximum is N*maxfreq
So, if the most common word had a relative frequency of 20%, then the number of word types would also be roughly 20% of the corpus size. I think this will overestimate pretty much always.