I am not sure if this is the right place to post or if I should be posting CompSci so please feel free to redirect me to the most appropriate StackExchange.
I would like to determine interests or hobbies that people have given two sources of data from their Twitter data. Those two pieces of information are their Bio, which is a user supplied description of themselves and their tweets (both a maximum of 140 characters).
I don't have background in NLP but intuitively I have been thinking of three contexts for determining interests given here with decreasing confidence:
- Using explicit word phrases that express interest: interested in, passionate about, etc.
- A positive sentiment expressed about the subject.
- Neutral or even negative sentiment for a topic (for example, somebody may be interested in their football team but dislike their current manager).
The result I am trying to achieve is to automatically label a user with a collection of interests such as:
And/or more specific examples like:
- FOREX Trading
- Android Development
I have tried using LDA, specifically the topicsmodels package in R, using an individuals tweets as the corpus but I am not so sure about the applicability of the results. LDA gives a collection of terms associated with the topics it found. This requires some manual interpretation to distinguish what the interest is.
For example you might get some results from LDA like the following:
- [Twitter, Java, time, work, day]
- [code, IPhone, mobile, look, job]
- [football, fantasy, team, pick, score]
I can generally pick out the topics people are interested in here but its just the key word or topic identifier I am interested not the other contextual terms like team, pick or score in the last example.
Perhaps part of speech tagging would be useful, maybe I should filter by noun, proper noun, proper name to get the core interest?
Also I would need to handle ambiguous topics such as Java, which could be a programming language, coffee variety or the island itself.
Any suggestions, references, links or pointers would be appreciated. I am comfortable with R, Java and Python if there are libraries which could help achieve this.