I am writing a tokenizer for a small NLP project. After the first attempt (basically splitting at white spaces) didn't lead to good results at all, I am now starting from scratch.
I would like to be able to detect names like "San Francisco" or "New York" as one token. My best guess is that I would need a list to check against for that? Does such a list exist somewhere on the internet?