3
votes
Accepted
Text classification of semantic relations within text
There are two paths you can take to solve this problem, both outlined in Jurafsky & Martin (2016), Ch. 15.
1) If computing your own semantic similarity matrix, you can shrink the size of the ...
- 576
3
votes
Text classification of semantic relations within text
Your broader question is about semantic relations, but the example you gave was about a narrower, but still widely researched field, that of opinion mining. Since you're asking about an entire field, ...
- 3,669
3
votes
Accepted
Differences in Naive Bayes Classifier NLTK, same code, different answers?
In the first case, you're training with the entire data set, then testing on different chunks of it.
In the second case, you're training with chunks of the data set, then testing on the whole thing.
...
- 68.2k
3
votes
How to tag_pos in nltk for a language that is not English?
Chapter 5 of the online NLTK book explains the concepts and procedures you would use to create a tagged corpus.
There are several taggers which can use a tagged corpus to build a tagger for a new ...
- 716
3
votes
How to search Penn TreeBank for arbitrary patterns?
There are quite a few tools which can do this.
For example:
Tregex mentioned by Jon Gauthier.
I know a few others:
PML-TQ https://ufal.mff.cuni.cz/pmltq/
Tundra https://weblicht.sfs.uni-tuebingen....
- 131
2
votes
Can anyone explain me the structure of the FCFG grammar
The NLTK book, chapter 10, which provides some theoretical background to the implementation, references NLTK ch. 9, where the feature grammar is introduced. Have you read these articles?
Ch. 9 ...
- 6,560
2
votes
Accepted
Implementation of Jaccard Distance metric in nltk.metrics.distance not consistent with the mathematical definition?
You are implementing the Jaccard coefficient whereas the library has the Jaccard distance. The coefficient tells how related two sets are (it is high when they are similar), whereas the distance does ...
- 4,495
1
vote
Accepted
What is the more natural parsing, the one that leads to the preferred reading of the sentence
For your first two sentences, my intuition is that the attachment of the PP subtly alters the semantics with the result that in some circumstances only one attachment is appropriate. We start with ...
- 101
1
vote
Jiang & Conrath when two words are the same
You can rewrite Jiang & Conrath similarity by: 1 / log(jcn_distance). If jcn_distance is equal to 0, then by adding 'log' the result will be 0.
- 1,535
1
vote
Accepted
Why are three tags necessary for the IOB format? What problem would be caused if we used I and O tags exclusively?
This blogpost on LingPipe gives a fine overview over chunk encoding. It says explicitly over the IO encoding (that is IOB without B)
The simplest encoding is the IO encoding, which tags each token as ...
- 30.4k
1
vote
Accepted
What are the most relevant ways to store expressions in NLP?
Yes, from the perspective of that manual the second answer is the right one.
2.1 Lists
What is a text? At one level, it is a sequence of symbols on a page such as this one. At another level, ...
- 2,540
1
vote
How to extract rows with only meaningful text in a column
Define “meaningful”. Implement that definition by means of a filter that will accept meaningful text, i.e., text that can be understood under such rules.
A modern way to solve this problem would be ...
- 456
1
vote
Accepted
How to determine who is the author of a message in the nps_chat corpora?
As stated here, the corpus is a bunch of XML files in which the authors are encoded as an attribute value to a post element:
To view the individual XML files in an editor (because this will help you ...
- 6,560
Only top scored, non community-wiki answers of a minimum length are eligible