2

I am working on a Sentiment-Analysis/Opinion-Mining of Tweets, focused on Finance related tweets.

One of the biggest issue I am facing is the unability of my algorithm to detect equivalent entities (Definition in B.Liu 2012: Page 18-19) when Financial slang is used. For example, for those familiar with it I would like the following entities to be detected as equivalent after lemmatization :

  • Govies = Government-Bonds = Sovereign-Debt
  • Cash = Monetary
  • Stocks = Equities
  • FX = Forex = Currency-exchange = Foreign-Exchange
  • Bund = German-Bonds = Bundesbank 10y
  • T-Notes = US10 = Treasury-Notes = US-Govies = American-Sovereign-Debt
  • Etc...

Here are my two questions :

  1. I was thinking about using some supervised learning (Naive-Bayesian-Classification) for such task, but can't find any classified set of data for training. Do you know if such dataset exists?
  2. Do you have any alternative idea regarding how to perform such task?

Thanks.

  • A good hint on whether two expressions are synonymous is whether they occur in the same contexts, i.e. once you have a corpus, you can easily check for collocations (n-grams of surrounding words) and the more of such contexts two expressions share, the more likely they are able to be used interchangeably and thus synonymous. This can very esaily be done with NLP tools, such as Python's NLTK library. – lemontree Aug 4 '16 at 17:47
  • Thanks @lemontree, I actually went for a word2vect approach that worked perfectly. – ylnor Apr 11 '17 at 12:04
0

The best answer was provided on a similar question on 'Cross-Validated',recommending to use the word2vec algorithm.

I obtained great results by fitting the model on a very large dataset of financial tweets and specialized financial-newspapers articles.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.