Do you know any cases of corpus linguistics tools or corpora, using JSON to "mark" or delimited linguistics units in morphology, syntax,semantics? It seems all marking systems use some kind of XML or derivative to mark text. I would like to know if there is an initiative using JSON instead.
I have seen a lot of corpora and corpus tools, but none using JSON. The only alternative to an XML based format is plain text coming in two flavours
- vertical format: One token per line with annotations separated by TABs
- inline format: plain text with annotations appended to each token like this: plain/ADJ text/NOUN with/PREP ... (the seperating character may vary, and there can be more than one layer of annotation)
There are several promising new developments where JSON is being used to provide improved modern data formats for computational linguistics, NLP, and corpus research.
There is an extensive project under way to create a JSON framework for linguistic fieldwork resources, including lexicon, texts, etc:
Hieber, Daniel W. (2018). Data Format for Digital Linguistics.
Check it out at https://format.digitallinguistics.io/
Or see the project on GitHub: https://github.com/digitallinguistics
Another useful source for JSON for syntax and semantics representations is:
Forbes, Angus G., Lee, Kristine, Hahn-Powell, Gus, Valenzuela-Escárcega, Marco A. & Surdeanu, Mihai (2018). Text Annotation Graphs: Annotating complex natural language phenomena. In Goggi, S. & Mazo, H. (Eds.), Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). Miyazaki, Japan: European Language Resources Association (ELRA). https://www.aclweb.org/anthology/L18-1169
Information about this JSON approach is available from Angus Forbes' Creative Coding Lab on GitHub:
Ines Montani of Spacy makes an interesting proposal for a new approach to JSON formatting for linguistic data/NLP:
Basic documentation for Spacy's (existing) JSON NLP format is here:
Spacy (2018). JSON input format for training.
Stanford NLP describes a JSON formatter: https://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/pipeline/JSONOutputter.html
Finally, at the Rezonator project (https://github.com/johnwdubois/rezonator) we are working to create a JSON format for the 2nd edition of the Santa Barbara Corpus of Spoken American English (SBCSAE). This is work in progress; stay tuned.