2

Disclaimer, I have no formal background in linguistics so I'm really asking in the dark here.

Problem: I'm trying to write a program that, given a grammatically correct (this is assumed) sentence, lists out all the grammatical patterns used in the sentence. My goal is to do this for multiple languages but I'll focus only on English for now.

Example:

Input: I have been studying for the exam but I am still nervous.

Output:

  • Present perfect continuous (S + have been + v)
  • Clause conjunction (C1 + but + C2)
  • Present be verb + adjective (S + be + adj.)

My hunch on the solution:

  1. Create tree traversal algorithm for each grammar pattern
  2. Tokenize the sentence and generate the (most likely) syntax tree and PoS of each token
  3. Run each traversal algorithm on the tree and save matching patterns

I honestly can't gauge how easy / difficult this question is. Some points that bug me are:

  1. How practical is it to list out all the grammatical patterns of a language?
  2. Will the running time be within reasonable bounds? (< 5s?)
  3. Are there any intractable edge cases?
Improve this question
3
  • 2
    First, get a list of all (not just a few, but all) the grammar patterns of English. Here's a list to begin with; it doesn't have all of them, but it's a good start.
    – jlawler
    Commented Aug 7, 2023 at 14:52
  • 1
    As for the point which are bugging you: 1. Not practical, as there is no comprehensive list, and if there was, it would be enormous. Also, language is often irregular and does not comply withh fixed patterns 2. No, as there will be many ambiguous cases which are slow to resolve (or impossible) 3. Yes, plenty.
    – Oliver Mason
    Commented Aug 8, 2023 at 10:54
  • 1
    You write, "ALL the grammar patterns used in a given sentence". In natural language, there is the cognitive reality. An example from Polish: Co robisz? Czytam. Translation: What are you doing? Reading. Another: Co robisz (z tymi póbkami)? Czytam. What do you do (with these samples)? I read (them). There is no change in the Polish form, it would be yet quite mad to ignore the cognitive reality, same as the Progressive and Simple in English. In short, to say "all" is challenging in linguistics. Natural language is generative. It is an infinity. "Form" and "pattern" are different potentials.
    – user13231
    Commented Aug 8, 2023 at 23:24

1 Answer 1

Reset to default
1

Your hunch is not that bad, essentially this is the approach of modern tools used in computational linguistics.

Tokenisation is generally seen as an easy task where word forms and punctuation marks count as tokens. The main thing is maintaining a good list of abbreviations and correctly recognising them in order to avoid wrong sentence ends.

Part of Speech tagging and morphological analysis are standard tasks in computational linguistics and there are good tools for that (at least for the English language), including free software.

For the syntactical analysis there are so-called parsers that come in two flavours, constituency parsers and dependency parsers. The former are just relating the constituent tokens of the sentence to each other and assign labels to the dependency relations, so they are sufficient for the task in the question, the latter analyse the sentence in terms of verb phrases and noun phrases and some other stuff. Again, there are freely available parsers of both kind at least for English.

The most successful PoS taggers and parsers nowadays aren't rule based but based on some machine learning algorithms and trained with human annotated "gold standard" data.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.