Working on ASR models, I have encountered several datasets which have distributions where a small amount of speaker make a huge part of the actual dataset.
The following image shows the extracted time spoken (log) per speaker from the Voxforge (de) dataset:
Extreme cases where the top 2% of speaker make up over 50% of the time spoken are possible.
My question is whether, or how much, this may impact the model's performance. How important is it to balance the dataset and which factors are most important (age, gender, total time spoken, etc.)?