I am doing a text classification task. My text is questions and answers in a community question answering website. I want to extract tags from the title , existing tags, and BODY of the questions and then use supervised learning to learn the tags for each class/category.
There is lot of noise in the BODY (punctuation marks, fully qualified names of Java exceptions, logs etc). How can I deal with such noise to extract good tags from them. Will some thing like Lucene or OpenNLP or any other library serve my purpose. And what techniques should I use.
How to capture intention of the user typing the question in NLP. Like for example, Robert understood my question and decided that its programming one and not linguistics. That is the problem I am trying to solve. How to automate that natural language understanding of text in community question answering sites?