So first, a disclaimer: I'm less than beginner level in everything that has to do with programming. I'm analysing the occurrences of English code-switching/translanguaging in a particular Polish-language IM groupchat. I've already compiled the conversation into a single text document (like 700k words in total, I know, a lot.) and now I'd like to extract just occurrences of English words among all the Polish words. How do I do it? I'd be super grateful for any beginner level explanations. p.s. I managed to upload the data to sketchengine too.
1
-
1What I would do is use a wordlist as the frontline approach, and then for words that aren't matched by either wordlist or, even, are matched by both, I would use one of the many trigraph-based "language detection" libraries around. These tend to work much better with text longer than just one word, though, so I'd definitely do #1 first and #2 only if that fails.– LjLNov 26, 2019 at 23:14
Add a comment
|
1 Answer
It seems like you are about to make a little corpus analysis. You already gathered different discourses together and created a corpus. I would recommend to use https://www.laurenceanthony.net/software/antconc/ It is a free professional software which I still use when analysing my own corpora. It is not that difficult to use and analyses a lot of things.
First of all, you should list all the words that occur in your corpus. Just ignore all the words that are not in English. As the next step you should think about what you are really interested in. "...occurrences of English code-switching/translanguaging in a particular Polish-language IM groupchat." Think about a hypothesis and then go on with the actual analysis.
Now you should look at collocation and stuff like this. So look up the words you found in the program and look where exactly they occur. What is there purpose and why does the speaker use them at this specific point. In Antconc this is pretty easy. You simply type in the word you want to analyse under the concordance tab and it will show all the passages of your corpus where they occur. After this it is up to you what exactly you want to look at.
And by the way, code-switching and translanguaging are not synonyms for each other. They are concepts about completely different things. I believe that you mean code-switching where you switch inside of a conversation to another language. Translanguaging is something different and should not be fortaken for code-switching. Furthermore you are also not comparing corpora. You are simply analysing a corpus.
-
Hey, thanks for the detailed answer!!! But the thing is that among the 700k words, only a fraction will be in English, so I'm trying to extract these ones. Doing it by hand will be extremely tedious... I do have another corpus with a collection of hundreds of thousands of words from English-language Twitter (I'd use a dictionary but I want to account for slang words/pooossibly potential typos as well), so I was thinking to extract all the words from the Polish corpus that are also present in the English one. Does that make sense? Again, thanks so much! Nov 27, 2019 at 17:27
-
If you work with Antconc this is pretty easy. Just make a word list of words you want to investigate on or words that you want to ignore in the word list tab. This is also a great way to learn how language functions. Otherwise you can also make a Zipf´s law distribution. At the far end you should encounter all the words which are no polish because they do not occur that often. Then you simply create a .txt with these words and load them into antconc or other corpus analysis tools. This is the professional way of analysing text.– JannikNov 28, 2019 at 12:01