sentiment analytics

Question

I've recently been tasked with programming a web crawler that searches through news articles for certain keywords--company names, for example--and then tries to determine if the article has a positive or negative slant. The web crawling portion is trivial but I feel that I've bitten off more than I can chew for the textual analysis portion.

My current approach is detailed below. I'm wondering if it's a sensible way to do things, or if there are better methods to try.

The crawler starts off by pulling the page's DOM. It then assumes that the actual "article" is within the paragraph <p> tags, so it reduces the data down to just that text. (This may be refined later on a per-website basis, but for now it works reasonably well). There are three input files: the aforementioned keywords, with words like "Facebook" or "Google", and then two wordlists, one containing positive words and the other containing negative.

The program then crawls through each paragraph of text in search for the keywords. If it finds a keyword, it starts checking every word against both word list. A keyword has a weight for each webpage, starting at 0, and incrementing for positive words found in the paragraph, decrementing for negative words found. So if 5 positive words and 3 negative words are found accompanying a keyword, a weight of +2 is assigned. If the combined weight of all paragraphs is positive (or even above some cut-off) the article is deemed to view it in a positive light, and the opposite for a negative score.

Obviously this approach brings some pretty huge issues. For example, a keyword (always a proper name) can be used once in an article, and the rest of the text may just refer to it as "it". I could make the scoping of the keyword per article (i.e., if the keyword is found anywhere in the page, then any paragraph containing listed words count toward the weight) but that brings it's own problems.

I'm not a linguist, and I realize this problem is probably pretty high up as far as linguistic problems go. I'm not looking for a perfect solution, but I'd like to know if there is anything more I could be doing.

Thanks for any feedback. I'm not sure if this is really appropriate for this stackexchange site, if it's not I'd like to know a better place.

Answer 1

Sentiment Classification is a still developing field so I don't feel comfortable saying that there is a right or a wrong way to approach your problem. Your approach would theoretically work but it is a bit simplistic. Consider the following:

First, how will you choose your word list? Will it be manually compiled? How will you make sure it is representative and has wide enough coverage to actually be useful?

Second, how will you choose your coefficients? From your description, I'm assuming all coefficients will be +1 or -1, but surely a word like "excellent" conveys more positivity than a mere "good" and reflecting this in your coefficients would improve accuracy. How will your coefficients reflect this? How do you know they are accurate?

I would suggest you read a few papers on Sentiment Classification (perhaps starting with the Google Scholar link I gave at the beginning of my answer) to get a general understanding of modern techniques.

From what I'm seeing, most solutions use some manually annotated sentiment corpus, like the Multiple Perspective Question Answer (MPQA) Corpus and perform machine learning techniques (usually SVM or MaxEnt) to decide which words and features are strong indicators of sentiment and what their optimal coefficient should be.

There are lots of machine learning libraries to help you with the programming component. A simple Google search should help you find a suitable language for your programming language of choice.

Answer 2

This book by Pang and Lee is a classic reference on the topic. To get a good overview of the topic, look at the book's website which has links to presentation slides.

I see that you want to implement the software on your own, and you want to (re-)invent algorithms too. The reasons behind these is not entirely clear. If you are writing a research paper or a thesis on the topic, you would need to do both. Otherwise, you can (1) look for free software for sentiment analysis; if you can't find any, or the licenses are unsuitable for your use, (2) learn techniques from published papers that demonstrate good results for the kind of classification you wish to do. There are a few workshops on this topic, and the website keeps changing from year to year, based on who is hosting it. In general, you could look for papers at "Workshop on Computational Approaches to Subjectivity and Sentiment Analysis" (WASSA-13, for example) or at "Practice and Theory of Opinion Mining and Sentiment Analysis" (PATHOS-2013, for example).