I'm especially interested in parsing sentences written in English (dependency or constituency). Ideally I'd like to have an estimation of how much improvement solving each of those major sources of errors would result in.
A nice study:
Kummerfeld, Jonathan K., et al. "Parser showdown at the wall street corral: An empirical investigation of error types in parser output." Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, 2012.
Constituency parser performance is primarily interpreted through a single metric, F-score on WSJ section 23, that conveys no linguistic information regarding the remaining errors. We classify errors within a set of linguistically meaningful types using tree transformations that repair groups of errors together. We use this analysis to answer a range of questions about parser behaviour, including what linguistic constructions are difficult for state-of-the-art parsers, what types of errors are being resolved by rerankers, and what types are introduced when parsing out-of-domain text.
(and kudos to the authors for providing the source code)
Another nice overview from Michael Collins given in his MOOC on NLP (lecture Week 4 - Lexicalized PCFGs > Completed Evaluation of Lexicalized PCFGs (Part 2) (11:28)):
Strengths and Weaknesses of Modern Parsers (Numbers taken from Collins (2003)):
- Subject-verb pairs: over 95% recall and precision
- Object-verb pairs: over 92% recall and precision
- Other arguments to verbs: 93% recall and precision
- Non-recursive NP boundaries: 93% recall and precision
- PP attachments: 82% recall and precision
- Coordination ambiguities: 61% recall and precision