All Questions
692 questions
33
votes
6
answers
19k
views
What are the fundamental differences between Natural Language Processing and Computational Linguistics?
I have a vague knowledge regarding those two fields, but I admit there are some fundamental concepts that I lack.
So, if we had to write down the actual differences between these two fields, what ...
- 9,493
30
votes
3
answers
7k
views
Why don't you get back the original text when you use translation software to translate something into another language and then back into English?
I translated "How to use Web?" into some other language using machine translation, but after I translated the result back into English, it is not the same as the first text I put into the translator.
...
- 395
26
votes
4
answers
4k
views
Is there a linguistics equivalent to Turing completeness?
In computer science, programming languages can be described in terms of "Turing completeness", basically, whether a programming language is capable of expressing any* algorithm. A non-Turing-complete ...
25
votes
6
answers
9k
views
Why is constituency needed, since dependency gets the job done more easily and economically?
I do dependency grammar (DG), and my personal view is that dependency gets the job done more efficiently than constituency by far. The average constituency parse (= phrase structure parse) contains ...
- 5,805
25
votes
5
answers
659
views
What has NLP/CL brought to the table of pencil-and-paper linguistics?
What role do NLP (natural language processing) and/or CL (Computational linguistics) play in today's theoretical linguistics?
Does, for instance, computability and formal specification play a big ...
user47
24
votes
7
answers
26k
views
Convert audio recording of word to IPA representation
Are the any open source tools/software libraries to convert an audio clip to its IPA representation? If so, are they accurate? If not, why not?
Here is a Gaelic word I wish to convert:
Ogg format:
...
- 1,072
18
votes
7
answers
6k
views
Looking for a good beginners reference to learn computational linguistics
Recently in my work I came across the Backus–Naur Form (BNF), one way of describing a context-free grammar. Since then, I've been interested in learning how to deconstruct and parse not only computing ...
- 517
18
votes
2
answers
3k
views
Is there a favoured data structure for storing ambiguous parse trees in Natural Language Processing?
I know a bit about parsing computer languages. Generally they try to resolve all ambiguities when parsing or abort the parse and throw an error.
This means you either got nothing because there was an ...
- 14.8k
18
votes
1
answer
13k
views
What do WordNet::Similarity scores mean?
I am using WordNet Interface in NLTK, which facilitates computation of a number of similarity metrics:
Path similarity
Leacock-Chodorow Similarity
Wu-Palmer Similarity
Resnik Similarity
Jiang-Conrath ...
- 283
17
votes
4
answers
42k
views
Automated French/Italian/German to IPA transcription
I'm looking for a website or software that will take text written in a source language and produce a transcription in IPA. The languages I am interested in are French, Italian and German, but if you ...
- 273
17
votes
5
answers
4k
views
Which language would be easiest for a computer to parse?
I have an idea for a programming language that would work more like a spoken language. "sentences" would have an initial context in which specific subjects, verbs, and objects would have meaningful ...
16
votes
3
answers
1k
views
Is there practical application of X' theory in natural language processing?
I was taught X' theory in generative grammar lectures, only to find out that NLP uses mainly simpler grammars, such as HPSG or statistical models such as hidden Markov models.
Is X' of any use for ...
- 870
16
votes
2
answers
4k
views
Is there a well-established metric to measure the effectiveness of a parsing algorithm?
My understanding that 100% accurate parsing (analyzing a text and creating a syntactic tree) is an impossible task for computational linguistics at this moment. However, there are many heuristics or ...
- 8,571
16
votes
3
answers
10k
views
Automatically Converting Natural Language to First-Order Logic
I'm trying to find a way to automatically convert arbitrary natural language sentences into first-order logic predicates. Although complex, this seems to be feasible to me, through inverse lambda ...
- 271
14
votes
2
answers
6k
views
What does a computational linguist actually do?
Currently, I working on my application for the Bachelor of Philosophy program at Penn State to major in cognitive linguistics, but at I am also a computer science major, my advisor wants me to ...
- 1,965
14
votes
4
answers
6k
views
Are there natural languages that do not obey Zipf's law?
Is there a natural language which is known not to follow Zipf's law? I'm interested to see if it's really universal.
This is what Zipf's law states:
Zipf's law states that given some corpus of ...
- 510
14
votes
2
answers
544
views
How do computational linguists abstractly represent a language?
When building models of the evolution of languages or similar phenomena where many different languages are involved and change over time, how do computational linguists abstractly model a language?
...
- 1,793
13
votes
3
answers
356
views
How do purely statistical machine translators deal with different word orders?
Even relatively closely related languages can differ greatly in word order. Take English and German for instance.
English is pretty boringly subject-verb-object whereas in German the finite verb must ...
- 14.8k
13
votes
1
answer
774
views
Automatic phonetic transcription of speech: is it still a difficult problem?
A friend was asking me why there aren't any programs that can analyze a user's pronunciation of words and then give feedback/suggestions.
Indeed, given an accurate phonetic transcription of a ...
- 131
12
votes
2
answers
4k
views
What is the entropy per word of random yet grammatical text?
It has been suggested that to construct an uncrackable yet memorable passphrase with 256-bits of entropy, the passphrase should be manifested as a poem.
The answerer made an estimation of the entropy ...
user4812
12
votes
1
answer
650
views
What are some resources that I can use to gather Twitter data for an NLP project?
I was going to use the Edinburgh Twitter Corpus that is referenced in this paper: The Edinburgh Twitter Corpus. But apparently Twitter has changed their Terms and the corpus is no longer available. ...
- 745
12
votes
2
answers
717
views
Bar-Hillel's critique of machine translation 50 years later
More than fifty years ago, philosopher Yehoshua Bar-Hillel wrote wrote an influential paper about computerized translation entitled: A Demonstration of the Nonfeasibility of Fully Automatic High ...
- 317
12
votes
4
answers
4k
views
Is there free and open Chinese corpus?
The corpus in NLTK sinica can be used for research.
There are two shortcomings for sinica to be used in research.
too small
it is a traditional Chinese corpus
Can someone here introduce a big and ...
- 223
12
votes
2
answers
458
views
If the Rosetta stone were to be translated today, would the process be the same as used the first time?
Are there any techniques available today that would make it considerably easier to anyone attempting to translate it today (as if they were trying todo it for the first time)?
Otherwise would the ...
- 221
12
votes
4
answers
600
views
Best method for building a learner corpus for DDL
I'm looking for a set of free and (somewhat) easy tools that I can use with my EFL writing students next semester. I want to analyze their initial essays and look for common errors that can be ...
- 301
11
votes
3
answers
2k
views
Efficient linguistic algorithms for detecting language of a website?
Some browser addons and web-services for website/dictionary translation sometimes offer a "automatic-language-detection" feature. This works more or less in my experience.
There is probably a variety ...
- 612
11
votes
2
answers
862
views
How to best clean a large historical corpus ridden with OCR errors
Overview:
I have a very large corpus of historical news papers (17th-20th cent.). The word count is about 20 bln. It's raw OCR-ed data in txt-files of about 150 GB. One newspaper issue per file (some ...
- 231
11
votes
2
answers
4k
views
Are there sentence boundary disambiguation algorithms which can handle punctuation errors with decent accuracy?
Most algorithms for splitting text into sentences which I've found rely on punctuation being correct. However, in many real world applications, there will be substantial numbers of punctuation errors (...
- 303
11
votes
1
answer
710
views
The power of trigram language models (2nd order Markov models)
Many people in computational linguistics seem to mention the unexpected power of trigram (or 2nd order Markov) models for language modeling. For instance, it has been stated (verbally) to me on ...
- 377
10
votes
2
answers
4k
views
Is there a list of word meanings that are universally represented in all languages?
I am looking for a comprehensive list of words/concepts that are represented in most if not all known languages - presumably the category would include human body parts (hand, foot, mouth, eye), ...
- 231
10
votes
3
answers
11k
views
How to test if a string of words is a grammatical sentence
Is there a way to test to see if a string of words forms a complete sentence?
For example:
The dog jumped over the fence == Good
The cat square seven the triangle == BAD
I was thinking the type ...
- 211
10
votes
6
answers
520
views
Does the P versus NP conjecture in computer science have any direct relevance to linguistics?
From Wikipedia:
The P versus NP problem is a major unsolved problem in computer science. Informally, it asks whether every problem whose solution can be quickly verified by a computer can also be ...
10
votes
1
answer
5k
views
How to identify character names in a body of text?
I'm trying to research methods of identifying or pattern matching names of characters in a novel or a general body of text, but so far my search has been unsuccessful, since "character" refers to ...
- 203
10
votes
1
answer
2k
views
Automatic/Computational Language Detection in Speech
Are there any packages that do much the same thing for audio/speech that the langid component/corpus of NLTK does for written text? The langid corpus/tool makes surprisingly accurate guesses about a ...
- 213
10
votes
1
answer
302
views
What statistical methods are used to test whether a corpus of symbols is linguistic?
In their 2009 paper on the Indus Script, Rao et al. describe a test for deciding whether a corpus of symbols is in fact a collection of texts in some language. Over-simplifying a bit, the approach ...
- 293
10
votes
3
answers
2k
views
Can parsing be classified to some complexity class (e.g. NP-complete)?
In computer science (especially computational complexity theory), problems can be classified to some complexity theory. For example, we say the travelling salesman problem belongs to NP-complete.
...
- 8,571
10
votes
2
answers
239
views
Linguistic research on translation evaluation
I have a background in natural language processing and machine translation, and recently I've been interested in automatic evaluation of translations.
I've read a lot of literature from the machine ...
- 181
9
votes
6
answers
3k
views
Is there a branch of linguistics using Calculus as a mathematical tool?
Is there a branch of linguistics using Calculus as a mathematical tool? I mean, can we use differential or integrate in linguistic study?
- 271
9
votes
3
answers
882
views
Mathematical preparation for postgraduate studies in Linguistics
I posted this question in https://math.stackexchange.com/ and it was suggested to me that it would be a good idea to submit the question here, too, as there might be more specialists on the matter.
I ...
- 243
9
votes
3
answers
9k
views
Non-Projective Example
I'm looking for an example sentence with a non-projective dependency parse.
It doesn't have to be in English, though such an example would be nice.
- 99
9
votes
2
answers
2k
views
Are there any statistics or web services for n-grams of frequent English words?
I found this for six common subjects. But it doesn't contain the complete statistics about all common English words.
- 233
9
votes
1
answer
652
views
What is the Relationship Between Document Length and Unique Words
Sorry if my question is a little more mathematical in nature, but my question is:
Suppose I took a document of some length whether it be news article, book, or something of that sort. What sort of ...
- 501
9
votes
2
answers
1k
views
IDE for context free grammars for natural languages?
Is there any such thing as a Integrated Development Environment for context free grammars (or any variant on a formal, machine parsable grammar) that target natural languages? The ones for BNF ...
- 2,924
9
votes
3
answers
4k
views
What's the real need for an end-symbol in n-gram models?
There's a footnote in Jurafsky & Martin (2008, p.89) pointing out that, without an end-symbol, an n-gram model would not be "a true probability distribution". Even after seeking the paper they've ...
- 191
9
votes
1
answer
530
views
What defines a unique writing style?
I'm an amateur writer that happens to be a professional programmer.
I say this because I've recently jumped back into a personal research project in which the goal is to automate the de-anonymization ...
- 191
9
votes
2
answers
496
views
Are linguistic corpora under threat from AI-generated text?
As many will be aware, there has been a lot of concern about the spread of AI generated text posted on the Stack Exchange network, leading to moderators on several sites standing down. See here and ...
- 4,118
9
votes
1
answer
249
views
Is there an automatic way of identifying transitive verbs in Computational Linguistics?
Is there any straightforward way of identifying transitive verbs (or sentences containing transitive constructions) in an BrE English text?
I've looked into semantic shallow parsers, such as Semafor, ...
- 377
8
votes
4
answers
2k
views
Machine translation - Rule-Based and Statistics-Based approaches
My question is about the current situation in machine translation.
I am aware about two main approaches to machine translation. One which is based in a strong way on linguistic theory and another ...
- 317
8
votes
4
answers
2k
views
What are the current hurdles to automatic audio to IPA transcription?
I have heard several people tell me that automatic segmentation and transcription to (narrow) IPA of fieldwork-quality audio is impossible at the moment, and even from laboratory-quality audio ...
- 282
8
votes
2
answers
203
views
Can computational techniques solve historical problems that couldn't otherwise be solved?
Recently I've read that machine learning has been used to apply the Comparative Method (example with references here). Also, there are other mathematical approaches that have been applied to the ...
- 713