0

This question is motivated by a question I read on another online forum, to which the answerers said that when they tested ChatGPT's Hindi, it made grammatical errors all the time and was also trash at word-choice. This is in stark contrast to my understanding of its performance in normal English, in which I don't normally find a mistake.

It made me think: maybe ChatGPT was trained on fewer Hindi texts.

The question that arose: has the connection been studied between how much text an AI bot is trained on for a certain language, and how accurately it can write original content in that language?

This would be analogous to observing second-language human learners and I imagine that it is a worthwhile thing to study.

(Context: I have no background in linguistics at all, but this is the best place to put my query.)

Thanks!

Improve this question
6
  • 4
    this has almost certainly been investigated, but imo is more of a computer science question than a linguistic one (such things are a common problem in machine learning, not just large language models), so should be moved to ai.stackexchange.com
    – Tristan
    Commented Jan 10 at 15:09
  • @Tristan this question involves testing a model's linguistic competence, which is not trivial. I don't know the AI.SE community so cannot judge whether they would be able to answer this for LLMs specifically, but I do think it is in scope here.
    – Keelan
    Commented Jan 10 at 15:37
  • Why would this be analogous to second language acquisition and not first language acquisition? Are you assuming specifically a model that is already trained for English receiving new training data for Hindi? (By the way, if you subscribe to the idea that humans have an innate language faculty, as many linguists do, LLM training is significantly different from both first and second language acquisition, because LLMs don't have this innate component.)
    – Keelan
    Commented Jan 10 at 15:45
  • 2
    @Keelan whilst the question of how to measure it is a linguistic one, the question that leads you to want to measure it (which is the question posed in the OP) is one of computer science, and general to all machine learning, not just LLMs. Likewise the literature on it is more likely to be in CS journals than linguistic ones (at least when examining all but the most easily accessible tools). That's why I think this question is more relevant to ai.se
    – Tristan
    Commented Jan 10 at 16:37
  • @Tristan that depends on what you consider "CS journals" and "linguistics journals". I am not a big fan of such strict demarcations. I only wanted to point out that I don't think this is off topic here (I am aware that you did not say that), but that there can be differences in the kind of answers you can get from different communities.
    – Keelan
    Commented Jan 10 at 17:53

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.