Questions tagged [computational-linguistics]

A branch of science that uses computers and mathematical methods to construct and investigate linguistic theory. Its technological and algorithmic implementation is called NLP.

Filter by
Sorted by
Tagged with
8
votes
1answer
2k views

What's the real need for an end-symbol in n-gram models?

There's a footnote in Jurafsky & Martin (2008, p.89) pointing out that, without an end-symbol, an n-gram model would not be "a true probability distribution". Even after seeking the paper they've ...
3
votes
1answer
1k views

Detection if input sentence represents an incomplete sentence?

I am curious if there is an automated way to detect if a given string, representing a sentence, is an 'incomplete' sentence. By this I mean that either the text is not grammatically correct, or is ...
1
vote
0answers
351 views

How do PCFG parsers actually work?

I'm new to NLP. I have a few doubts about PCFG parser (NLTK). My understanding is that PCFG parser will return most probable parse tree. So if I'm parsing one sentence with PCFG parser, I'll be ...
1
vote
1answer
912 views

Gold Part-Of-Speech tags

I've read this term in many papers in NLP: (Gold POS tags). what does it mean? Thanks.
1
vote
1answer
272 views

Evaluate idea to autobuild russian-english parallel corpus

I was looking for russian-english parallel corpus, but have found none publicly available to the moment. If you know one, or parallel text database, I would be very grateful for one. (I heard of ...
4
votes
1answer
507 views

What is the most comprehensive context-free grammar for English?

I realize how hard it is to develop a CFG to model an arbitrary English sentence. Some valid discussion points for this topic can be found here: Is there an EBNF that covers all of English? However, ...
1
vote
1answer
1k views

Steps to build a dependency parser [closed]

I'm a beginner in NLP. What are the main steps to build a dependency parser? thanks.
1
vote
4answers
206 views

Is it possible to build thesaurus automatically?

Is it possible to build thesaurus by simply analyzing lots of text, or it's obligatory should be handmade? Also a question: is there free-to-download&use English language thesaurus?
5
votes
2answers
4k views

Detecting actions within text

I'm looking for a way to computationally find any actionable clauses within a body of text. Ideally there is a library I could use that I just give a body of text and it spits out the actions that it ...
2
votes
2answers
210 views

Determine adjective order algorithmically

Does NLTK (or any other Python natural language module) have the ability to determine the order of multiple modifier adjectives? For example: metal round huge bowl (incorrect) huge round metal bowl (...
1
vote
1answer
437 views

What is the linguistic significance of dep and gov methods in Stanford NLP's Typed Dependency class?

I am looking at a TypedDependency object like this: root(ROOT-0, fired-4), which is part of the StanfordNLP suite. If I call the object's .dep() method I get fired/VBD If I call the object's .gov() ...
3
votes
3answers
248 views

Book suggestions for linguistics for Computer Science people

I am asked to do some NLP tasks on a language which is agglutinative. I am finding these terms difficult to understand since my background is different. I am looking for some nice books that give a ...
3
votes
0answers
143 views

What are the foundational papers in computational phonology?

I'm working on a paper on the history of computational phonology. As I understand it, Chomsky & Halle's SPE acted as a catalyst for research in the field, namely by laying the foundations for two-...
7
votes
1answer
745 views

Earley Parser: Ambiguity

I've got a pretty basic question concerning the Earley parser: In case of syntactic ambiguity ( S -> NP VP(V NP(NP PP)) vs. S -> NP VP(VP((V NP) PP) ), are both parses stored in one chart or in two? ...
4
votes
1answer
384 views

Combine dictionary lemmatization with stemmer

I'm doing lemmatization as part of a classifier right now. My question is, if it makes sense to first use a dictionary lookup, to find the lemmas (wordnet) and after that additionally, to apply a (...
2
votes
1answer
349 views

Can the entropy per word be caculated precisely?And relation among information theory, semantics, and pragmatics

What we have gotten about the expected per word entropy of random yet grammatical text is just some upper bound of the the expected per word entropy, because we have not found the exact way to compute ...
4
votes
2answers
588 views

dependency or constituency treebank?

Most Treebank conversion which I found in the web are from constituency treebank to dependency treebank, I wonder why there is little works in the opposite direction? Does it mean that nowadays ...
7
votes
2answers
951 views

Do I have copyright issues when making a corpus from the web?

I realize that this is probably technically off-topic as it relates to copyright legalities of corpus construction. However, it is a practically very important issue for corpus linguistics and so I ...
3
votes
1answer
230 views

VerbNet semantic roles and preposition groups - how to determine matches

Using verbnet to test whether a sentence matches a frame, how does one determine whether the semantic role specified in the verbnet frame is appearing in the sentence or not? e.g. on this verbnet ...
3
votes
0answers
480 views

Stanford NLP parsers and idioms that have common semantic meaning

I have parsed the following sentence in the Stanford CoreNLP demo page and the Stanford parser demo page. Although both result in a parse that can imply purpose semantics (hinging on the advcl and the ...
1
vote
1answer
210 views

What is state of the art in the field of NLP? [closed]

I am new to Natural Language Processing, I think NLP is a challenging field, the syntax and semantic ambiguities could cause a lot of problems. For example I think for these problems machine ...
3
votes
0answers
67 views

Any NLP performance timeline?

I am looking for any plots, etc. that show performance metrics on various NLP tasks as a function of year. E.g. the following figure shows the evolution of performance of face recognition systems:
3
votes
2answers
150 views

Center of a set of words

Is there any available algorithm that can take a set of words and attempt to find a word that best represents the "center of mass" of all those words? This would be easy if we can define a distance ...
5
votes
3answers
202 views

Has anyone attempted to simulate the creation/development of a language?

I suspect this is a dream confined to sci-fi for at least a few decades, but, I'd like to know if anyone has ever tried to create a computer model for the creation of a language, or at least ...
17
votes
1answer
12k views

What do WordNet::Similarity scores mean?

I am using WordNet Interface in NLTK, which facilitates computation of a number of similarity metrics: Path similarity Leacock-Chodorow Similarity Wu-Palmer Similarity Resnik Similarity Jiang-Conrath ...
0
votes
0answers
73 views

NLP system with end-to-end ambiguity support?

I've noticed that most NLP systems I can easily investigate on the Internet don't handle ambiguity at all, or only handle within components of the pipeline and not between components. For instance it ...
1
vote
1answer
362 views

NLP conversion between parts of speech and pertainyms?

I would like to write a program that can automatically group e.g. 'happiness', 'happily', 'happy' into 'happy': What do I need to read to get a handle on this subject? What is it called? What is the ...
0
votes
0answers
84 views

adjective features - similar to beth Levin's verb classes?

Beth Levin published research on verb classes/alternations - is there similar for the adjective category of words? I can see references to countable/uncountable, gradable,absolute, of quantity/quality ...
7
votes
2answers
566 views

Finding the root of an Aramaic or Hebrew word

I'm trying to make a dictionary on the web where people can automatically look up words in a text they're reading online. While I have the words and their definitions, the problem is that many times ...
2
votes
5answers
496 views

Introduction to linguistics from a practical application standpoint

I currently have a project that I need to do that involves quite a bit of computational linguistics. My background in linguistics is very high level - I wonder if there are any good introductory ...
0
votes
1answer
129 views

Switchboard queries with NXT (the TEXT function)

The NXT query language includes the query function TEXT, which should return the textual content of the variable it applies to. In theory, Switchboard should support this. The file README.SWBD-QUERIES....
0
votes
1answer
2k views

Are there any words that have changed usage/meaning multiple times (multiple semantic shifts) from 1850s to present time [closed]

I am interested in semantic shift undergone by words, and I am aware of classic examples like 'gay', which has shifted in meaning since the 1900s. However, are there any words that have actually ...
4
votes
1answer
2k views

Why are functional/logic programming languages like Prolog or Haskell popular with computational linguistics and AI?

This may be the wrong place to be asking this question, but I'm not entirely sure what kind of advantage using a language like Prolog gives you over an object-oriented language like Python. Is there a ...
1
vote
1answer
359 views

Looking for a Morphology Database

This is my first StackExchange post, so please let me know if I need to add additional information. I am looking for a comprehensive database that has words broken down by their morphology. For ...
3
votes
2answers
366 views

Probabilities for 2-grams are higher than 1-grams in arpa file produced by kenlm

I'm using the 1 billion word language corpus to build a model with 1 and 2-grams. When using the lmplz program that comes with kenlm, I noticed that the arpa file seems to have higher probabilities ...
11
votes
2answers
3k views

What is the entropy per word of random yet grammatical text?

It has been suggested that to construct an uncrackable yet memorable passphrase with 256-bits of entropy, the passphrase should be manifested as a poem. The answerer made an estimation of the entropy ...
4
votes
2answers
486 views

Which dependency parsers take auxiliaries to to be heads over content verbs and prepositions to be heads over their complements?

The Universal Stanford Dependencies (USD) parsing scheme subordinates auxiliary verbs to content verbs and prepositions to their complements: http://nlp.stanford.edu/~manning/papers/...
1
vote
2answers
1k views

Noise Removal -Text classification

I am doing a text classification task. My text is questions and answers in a community question answering website. I want to extract tags from the title , existing tags, and BODY of the questions and ...
1
vote
0answers
179 views

Punctuation and spaces in other common languages?

For example, in Chinese, a full stop is 。 (U+3002) and a space is   (U+3000). I need to break paragraphs into sentences and I want to make it compatible with as many languages as possible. What other ...
-1
votes
1answer
53 views

“Rate” or “score” a word by various parameters

I’m working on a small project which involves a large list of words (single words, no sentence) and I would love to have a way to “rate” or “score” each word by some parameters. Right now I have two ...
1
vote
1answer
168 views

recognizing if a message is question or not?

Although there are appreciable comments and an answer after this question (thank to Robert and Acattle), I am much seeking for a Knowledge branch, (e.g. an "NLP sub-task") than an individual research: ...
2
votes
1answer
196 views

Spelling Correction on Romanized Devanagari texts

Has anything been done in this regard? I am looking for research that has been conducted for spell checking of Romanized Devanagari text. I suppose well researched English spell checking algorithms ...
3
votes
3answers
2k views

How to determine whether a given sentence is demanding an answer or is providing some information?

I tried with basic thing like whether question starts with who/what/.. but there are a lot many sentences which do not start with interrogative words but still demands an answer like "hotels in ...
1
vote
1answer
704 views

Tokenizing first and last name as one token

Is is possible to tokenize a text in tokens such that first and last name are combined in one token? For example if my text is: text = "Barack Obama is the President" Then: text.split() results: ...
1
vote
2answers
1k views

Two sentences with the same meaning

I am looking for a statistical or linguistic method that could give the degree of similarity between the meaning of two sentences. I have found in literature many distance measures (euclidean distance,...

1
7 8
9
10 11
13