1

I'm doing a research on defining the complexity of language used in technical documentations for technologies (libraries and modules) used in data science and machine learning engineering. And I'm expecting that the results could help to develop English curriculums that will adapt for data science and machine learning students. The data that I have is this:

  • Part-of-speech tags
  • Dependency tags
  • Noun Phrases tags
  • Verb Tenses tags
  • Root form of words

    Using all this data, how can we determine grammatical complexity from a quantitative standpoint only?

    • Improve this question
    1
    • 5
      First task: define "grammatical complexity". Even if you think it's a meaningful concept, you definitely won't be able to calculate it if you ignore morphology.
      – curiousdannii ♦
      Commented Jul 27, 2020 at 13:55

    1 Answer 1

    Reset to default
    1

    There is a framework named CAF (complexity, accuracy, fluency) used to measure complexity in language learning and language understanding. However, it uses rather simple measures for complexity like length of sentences, T-units, or clauses.

    For something more sophisticated, you can look at dependency length. Note that differences in dependency length are only sensible for sentences of a fixed length, and make sure to have the same dependency parser for all of the sentences you compare, differences between parsers can completely invalidate any conclusion.

    Improve this answer
    2
    • Thanks Monica! I found that dependency length is a good way to go from, and I'll be using spaCy dependency parser. But, for more clarification, you're saying that dependency length are only sensible for sentences of a fixed length, can you make further clarification?
      – Arka-cell
      Commented Jul 30, 2020 at 20:54
    • 1
      I suggest that you plot depedency length (DL) vs. sentence length (SL) for your data an look at the curve. The DL grows with the sentence length, but the growth is not linear, so you cannot normalise DL by dividing it by SL..
      – Sir Cornflakes
      Commented Jul 31, 2020 at 6:24

    Your Answer

    By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

    Not the answer you're looking for? Browse other questions tagged or ask your own question.