Hot answers tagged

19 votes
Accepted

What do WordNet::Similarity scores mean?

The most accessible resource that explains the difference between each of these word similarity metrics would be Dan Jurafsky and James H. Martin's ubiquitous Speech and Language Processing 2nd ...
user avatar
4 votes
Accepted

How to search Penn TreeBank for arbitrary patterns?

Your question is tagged for NLTK, but if you're free to switch, Stanford NLP has a product called Tregex which does exactly this kind of tree search. You might also be able to shell out to the Tregex ...
user avatar
4 votes
Accepted

Why does Wordnet lack stopwords ("the", "of", "her", etc)?

WordNet is not a dictionary but a semantic lexical database. The key function of WordNet is to create a network of semantic relationships between words (synonyms, meronyms, etc.) So it makes sense it ...
user avatar
3 votes
Accepted

Differences in Naive Bayes Classifier NLTK, same code, different answers?

In the first case, you're training with the entire data set, then testing on different chunks of it. In the second case, you're training with chunks of the data set, then testing on the whole thing. ...
user avatar
  • 53.3k
3 votes

How to tag_pos in nltk for a language that is not English?

Chapter 5 of the online NLTK book explains the concepts and procedures you would use to create a tagged corpus. There are several taggers which can use a tagged corpus to build a tagger for a new ...
user avatar
  • 716
3 votes
Accepted

Text classification of semantic relations within text

There are two paths you can take to solve this problem, both outlined in Jurafsky & Martin (2016), Ch. 15. 1) If computing your own semantic similarity matrix, you can shrink the size of the ...
user avatar
  • 576
3 votes

Text classification of semantic relations within text

Your broader question is about semantic relations, but the example you gave was about a narrower, but still widely researched field, that of opinion mining. Since you're asking about an entire field, ...
user avatar
  • 3,537
3 votes

How to search Penn TreeBank for arbitrary patterns?

There are quite a few tools which can do this. For example: Tregex mentioned by Jon Gauthier. I know a few others: PML-TQ https://ufal.mff.cuni.cz/pmltq/ Tundra https://weblicht.sfs.uni-tuebingen....
user avatar
  • 131
2 votes

NLP conversion between parts of speech and pertainyms?

Based on your comment, you seem to be looking for positive words with the intent of performing sentiment analysis (generally lists are divided into positive, negative, and neutral). Here are some ...
user avatar
  • 417
2 votes
Accepted

Implementation of Jaccard Distance metric in nltk.metrics.distance not consistent with the mathematical definition?

You are implementing the Jaccard coefficient whereas the library has the Jaccard distance. The coefficient tells how related two sets are (it is high when they are similar), whereas the distance does ...
user avatar
  • 3,085
2 votes

Can anyone explain me the structure of the FCFG grammar

The NLTK book, chapter 10, which provides some theoretical background to the implementation, references NLTK ch. 9, where the feature grammar is introduced. Have you read these articles? Ch. 9 ...
user avatar
  • 6,150
1 vote
Accepted

What is the more natural parsing, the one that leads to the preferred reading of the sentence

For your first two sentences, my intuition is that the attachment of the PP subtly alters the semantics with the result that in some circumstances only one attachment is appropriate. We start with ...
user avatar
  • 101
1 vote
Accepted

Why are three tags necessary for the IOB format? What problem would be caused if we used I and O tags exclusively?

This blogpost on LingPipe gives a fine overview over chunk encoding. It says explicitly over the IO encoding (that is IOB without B) The simplest encoding is the IO encoding, which tags each token as ...
user avatar
1 vote
Accepted

What are the most relevant ways to store expressions in NLP?

Yes, from the perspective of that manual the second answer is the right one. 2.1 Lists What is a text? At one level, it is a sequence of symbols on a page such as this one. At another level, ...
user avatar
  • 2,218
1 vote
Accepted

Determine adjective order algorithmically

Do you actually need to come up with the properties for your adjectives, or is the goal to determine whether some order is valid (and properties are just a way of making that determination)? If the ...
user avatar
  • 126
1 vote

Semantic Relatedness metric across Parts of Speech

I don't know of a toolset, but I can think of a factors that you'd need to account for to determine 'relatedness' across parts of speech: First, consider for the eight parts of speech: verb ...
user avatar
  • 223
1 vote

How to extract rows with only meaningful text in a column

Define “meaningful”. Implement that definition by means of a filter that will accept meaningful text, i.e., text that can be understood under such rules. A modern way to solve this problem would be ...
user avatar
  • 436
1 vote
Accepted

How to determine who is the author of a message in the nps_chat corpora?

As stated here, the corpus is a bunch of XML files in which the authors are encoded as an attribute value to a post element: To view the individual XML files in an editor (because this will help you ...
user avatar
  • 6,150

Only top scored, non community-wiki answers of a minimum length are eligible