Often I want to make my own corpus of a new language I've become interested in.
I'm very happy to make my own tools and have plenty of programming experience. I have made my own tool to gather plain text random samples from a specified language edition of Wikipedia and it has worked pretty well.
But sometimes a language doesn't have its own Wikipedia, or its Wikipedia is too small or shows too many artefacts being heavy on articles on certain topics.
What I would like is a tool that crawl the web and gather pages using only a certain language.
It doesn't have to do anything linguistic, raw HTML is usable, plain Unicode text is better, but if it can also do things like word frequency, normalizing, lemmatizing, etc that would be a great bonus.
My current language of interest is Mongolian written in the traditional script, which does indeed seem to have a large enough web presence.
Is there a tool that could help me with this?