Questions tagged [statistical-mt]
The theory and practice of MT that uses probabilistic models and statistical decision procedures to generate optimal translations. Usually included is the task of multilingual text alignment.
16
questions
-2
votes
1
answer
154
views
Is large language model “perplexity” on an input the total of its perplexity for the smallest parts? [closed]
“Perplexity” is how likely a large language model scores an output it gives as being the output it should have given. If the model does not produce responses for itself to choose from that it ...
- 17
0
votes
3
answers
122
views
Is there a topic in linguistics that uses differential calculus or AP stats level statistics?
I'm doing a math project that requires the research of an application of differential calculus or statistics, and as someone who is very interested in linguistics, I was wondering if there is any area ...
- 11
0
votes
0
answers
21
views
Is there any theory for mathematically judging the quality of a parallel corpus before running it through a machine translation pipeline?
The performance of a machine translation system is highly impacted by the parallel corpus it is trained on. Therefore, as we all know, a good quality normalized and (ideally) noise-free corpus is ...
0
votes
0
answers
42
views
What do distributed frequency and raw frequency mean?
What do distributed-frequency, distributed-frequency cum-age, raw-frequency, and raw-frequency-cum-range mean?
They are mentioned on link:
"Second, young age group under 15 rely more on core ...
- 19
1
vote
0
answers
21
views
Direction of alignment function in statistical machine translation
I'm trying to learn IBM Models and statistical machine translation. I have read various resources and realized that in same articles, alignment function is designed in direction from target language ...
- 111
1
vote
0
answers
391
views
Simple bigram letter model
I am working through an exercise where, given a set of corpora, I will implement a simple model on a test corpus to determine the most likely corpus.
Say the corpora with which I want to learn are ...
- 19
1
vote
2
answers
136
views
What are some good books/MOOCs in field of computational semantic analysis?
I have recently started learning Natural Language Processing and while reading research papers I came across terms like "Latent Dirichlet Allocation", "Latent Semantic Analysis", "Gaussian ...
- 247
2
votes
1
answer
64
views
How lexical ambiguity addressed in phrase-based SMT
I am wondering how the lexical ambiguity is addressed in phrase-based statistical machine translation?
- 123
0
votes
0
answers
42
views
Phrase suggestion model using a gold standard corpus
There are already spell checking models available which help us to find the suggested correct spellings based on a corpus of trained correct spellings. Can the granularity be increased to "word" from ...
- 240
2
votes
1
answer
61
views
What statistical methods are appropriate to comparing e.g. VOT across different groups?
As part of forensic linguistics practice, I must assess the relative age of multiple individuals based only voice. One crucial variable is VOT, which is known to decrease with age.
So, let's say ...
- 2,701
3
votes
4
answers
347
views
"Episode 78" machine mistranslation etymology
This post on Rocket News 24 described a Google Translate bug, in which
さよなら大好きな人
was translated into "Episode 78". This bug is still live as of the time of writing, and you can replicate it ...
- 405
2
votes
1
answer
403
views
Can the entropy per word be caculated precisely?And relation among information theory, semantics, and pragmatics
What we have gotten about the expected per word entropy of random yet grammatical text is just some upper bound of the the expected per word entropy, because we have not found the exact way to compute ...
2
votes
0
answers
57
views
Generalisations which a bi-gram probabilistic model might infer from a dataset
I have the following exam question for a machine translation course:
From my understanding, I assume the answer is looking for incorrect English grammar which get discovered by bi-grams.
So the ...
- 171
4
votes
1
answer
130
views
Automated methods to align text
I have an assignment for my machine translation class which involves using the different language versions of websites as parallel corpora. The problem is the text isn't properly aligned.
For example:...
- 171
13
votes
3
answers
333
views
How do purely statistical machine translators deal with different word orders?
Even relatively closely related languages can differ greatly in word order. Take English and German for instance.
English is pretty boringly subject-verb-object whereas in German the finite verb must ...
- 14.6k
8
votes
4
answers
2k
views
Machine translation - Rule-Based and Statistics-Based approaches
My question is about the current situation in machine translation.
I am aware about two main approaches to machine translation. One which is based in a strong way on linguistic theory and another ...
- 317