5
votes
Accepted
Where can I find the letters of documented writing systems, as text, online?
Most well documented writing systems have been encoded in Unicode, at least to some degree. You can see a list of supported scripts here and code charts for each script here.
A few scripts (including ...
- 1,162
3
votes
Is a text with orthographic or grammatic mistakes in a language X still a text in that language X?
A useful philosophical framework for discussion is the classic dichotomy between competence and performance, promulgated by the theory of generative grammar, and the difference by I-language and E-...
- 80.3k
3
votes
How can I do data augmentation for text classification?
A method for introducing variation in language data is round-trip translation to a different language and back. Shalom Lappin used machine translation for this purpose (and noted that Google translate ...
- 29.6k
3
votes
Accepted
How can I do data augmentation for text classification?
NoiseMix is made for exactly this. (Full disclosure: I am advisor to the project.)
It applies simple transformations to rows of text data while preserving labels.
The word-level perturbations are:...
- 7,522
2
votes
What are the various approaches to detect whether a sentence is complete or not?
For formal English anyway you can parse with spaCy and then iterate through the tagged tokens looking for a finite verb (VerbForm=Fin, as opposed to Ger or Inf).
See https://spacy.io/usage/linguistic-...
- 7,522
2
votes
Accepted
What parts of linguistics deal with the differences between text types?
The features mentioned in the last paragraph of the question belong to the domain of corpus linguistics and stylometry.
For the categories mentioned in the first part of the question, Systemic ...
- 29.6k
2
votes
How can I do data augmentation for text classification?
Compared to images, data augmentation in NLP is tricky as even a slight change in the sentence can entirely change its meaning.
However, there are some augmentation techniques I have seen in the ...
- 121
2
votes
Is a text with orthographic or grammatic mistakes in a language X still a text in that language X?
Yes
If the sentence is intended to be in that language, then I'd say that it is in that language. Intent matters a lot in communication - for example, the utterance "Hallo!" would be a valid sentence ...
- 414
1
vote
Linguistic analysis of ChatGPT's default style of writing
This is such a fantastic question, I’ve struggled to offer even a small, direct answer.
Let us begin in the simplest possible way, before hopefully attempting some generality.
The easiest thing you ...
- 301
1
vote
Is a text with orthographic or grammatic mistakes in a language X still a text in that language X?
Yes, of course.
In corpus linguistics we often have to deal with texts digitised using OCR and containing some amount of OCR errors. We are completely aware that we cannot get rid of the OCR errors ...
- 29.6k
Only top scored, non community-wiki answers of a minimum length are eligible