What algorithms can associate words together from texts?

Question

What algorithms can associate words together from texts? I was wondering how this can be achieved. Thanks.

Answer 1

You could do it like this:

Go through a lot of tweets, for each tweet: For each word in this tweet, store the word (if not already stored) somewhere and put all other words in this tweet into a list that is connected to this entry. If some word comes up a second (third, ...) time, don't add it, but count +1 instead.

I wrote it for you in Python:

tweets = ["Trump is a loser",
          "What a loser man. Trump go home"]
frequencies = {}

for tweet in tweets:
    for word in tweet.split():
        remainder = [otherword for otherword in tweet.split() if not otherword == word]
        try:
            frequencies[word]
        except:
            frequencies[word] = {}
        for otherword in remainder:
            try:
                frequencies[word][otherword] += 1
            except: #no entry yet
                frequencies[word][otherword] = 1

for k,v in frequencies.items():
    print(f"\n{k}:")
    print("\n".join([f"{x[1]} co-occurrence(s): {x[0]}" for x in sorted(v.items(), key=lambda x: x[1], reverse=True)]))

Results in:

Trump:
2 co-occurrence(s): a
2 co-occurrence(s): loser
1 co-occurrence(s): is
1 co-occurrence(s): What
1 co-occurrence(s): man.
1 co-occurrence(s): go
1 co-occurrence(s): home

is:
1 co-occurrence(s): Trump
1 co-occurrence(s): a
1 co-occurrence(s): loser

a:
2 co-occurrence(s): Trump
2 co-occurrence(s): loser
1 co-occurrence(s): is
1 co-occurrence(s): What
1 co-occurrence(s): man.
1 co-occurrence(s): go
1 co-occurrence(s): home

loser:
2 co-occurrence(s): Trump
2 co-occurrence(s): a
1 co-occurrence(s): is
1 co-occurrence(s): What
1 co-occurrence(s): man.
1 co-occurrence(s): go
1 co-occurrence(s): home

What:
1 co-occurrence(s): a
1 co-occurrence(s): loser
1 co-occurrence(s): man.
1 co-occurrence(s): Trump
1 co-occurrence(s): go
1 co-occurrence(s): home

...

You'd have to deal with punctuation marks yourself, also with upper/lowercase.