Corpus with sentences translated to English

Question

Does anyone know of a corpus I could use in my research?, for any language, where each sentence is translated correctly into English. In particular, the corpus should contain primarily everyday spoken speech.

For example for Swedish:

Var är du? = Where are you?
Gjörde du det? = Did you do it?
Ge mig det! = Give me it!
Det är dax att åka nu! It is time to go now!

It would also be of interest to me if the language in question comes with a good Text-to-Speech synthesis system. For example, the Irish language has a good synthesiser:

http://www.abair.tcd.ie/?page=background&lang=eng

But I haven't found an Irish-English corpus for everyday spoken speech.

Answer 1

It’s called a parallel corpus. Searching online for that term, there are almost too many:

http://www.glottopedia.org/index.php/Parallel_corpus

https://en.m.wikipedia.org/wiki/Parallel_text

https://www.sketchengine.eu/corpora-and-languages/parallel-corpora/

https://conferences.unite.un.org/uncorpus

https://opus.nlpl.eu/

https://www.clarin.eu/resource-families/parallel-corpora

But I always recommend finding a direct method rather than overloading yourself with too many different sources to keep track of.

You can make your own parallel corpus with computational techniques.

There are various options as to where you will get your data from, and how you will collect and process it.

You can gather data yourself either by compiling it manually (like, working with speakers of a language to produce the data), or by extracting it from pre-existing sources such as with web scraping (from websites) or text processing (from texts and so on). You then need techniques to align language units (for example, sentences) from a corresponding sentence in the one language to the other.

If you want to make your own data, this is just an example of a resource explaining some of the techniques in that field:

https://www.researchgate.net/publication/220875011_Crowdsourcing_Translation_Professional_Quality_from_Non-Professionals

If you want to web scrape to build a parallel corpus, this article could give you an introduction to that:

https://www.taus.net/resources/blog/web-scraping-for-parallel-corpora-creation

and this Stack Exchange post:

https://datascience.stackexchange.com/questions/87317/build-a-corpus-for-machine-translation

And I assume there are also online resources of parallel texts, which can be a helpful way to make a parallel corpus, since you already have side-by-side texts, so it can be easier to pair one sentence from a first language with another in the second language.

A couple good general-purpose resources for trying to search for particular linguistic datasets or corpora are:

https://datasetsearch.research.google.com/

https://huggingface.co/docs/datasets/index

https://opendatahandbook.org/

https://www.kaggle.com/datasets

https://github.com/awesomedata/awesome-public-datasets

http://www.elra.info/en/

https://dataportals.org/

https://www.ldc.upenn.edu/

But basically, the simplest will probably be to google “language 1, language 2 parallel corpus” i.e. “German French parallel corpus”.

Answer 2

Tatoeba provides such a collection of sentences. It is created and maintained by volunteer editors and covers at the moment of this answer more than 11 million sentences in 420 languages.

Answer 3

There is a specific page for parallel data, with a curated list of data sets and data directories in machinetranslate.org: https://machinetranslate.org/parallel-data