What's the hardest language to learn???
I would say that undocumented languages with very few speakers are extremely difficult to learn. Even for children. An extinct language with no trace left must be the answer to your question.
All living languages with compact communities are reasonably easy to learn, the laws of evolution require that. A language that is too difficult to be learned (both by children and by adults) would be doomed.
I don't think that this is an answerable question. You would have to distinguish between "has acquired the language" from "has not acquired the language", and you'd have to be able to do it in a way that you're measuring the same thing across cultures / languages. In particular, you would have to distinguish between "has acquired the language" and "has mastered all aspects of our culture". My experience is that cultures differ substantially in the how important it is to know terms for now-obsolete bean-mashers, and to learn social rituals about the use of "whatever", so I think any test would be skewed by ideologies about what it means to "have learned" the language.
An approach to answering this is to compile statistics on the average age that children start to talk, learning as natives to speak the various languages of the world. The higher that age, the harder it must be to learn the language. The last I heard, by this measure, Russian seems to be quite difficult.