You have a corpus and training set.
- Build the weighted n-grams for each word in the corpus; [data1]
- Build n-grams for the words in the training set; It will be unweighted since each word would have low frequency; [data2]
- Apply word weights from [data1] to [data2];
- Compute sentence weights for [data2]; [data3]
- Sort [data3] descending;
N-gram is essentially a set of N items (words), which is associated with a Markov property. This property can be called "weight" or "probability".
Say, a corpus contains a sentence:
I ate a green apple
The bigrams for it would be: "I ate", "ate a", "a green", "green apple".
Note there can be funnier bigrams like "I ??? a", "ate ??? green", "a ??? apple".
The trigrams are: "I ate a", "ate a green", and "a green apple".
You have actually started moving this way. Computing a word frequency is essentially an unigram, a corner case of n-grams where the order of Markov chain equals to zero. A single word along with its weight is an unigram by itself.
Step 1, analyzing the corpus
Say, your corpus contains 30M phrases and 30 of them contain a word pair "green apple". The bigram would be like:
"green", "apple", 0.000001
30 / 30,000,000
The result data would be an array of such structures.
Step 2, analyzing the training set
Build the same data for the training set.
Your training set most likely does not contain a representative number of sentences. But this is not a problem since you already have weights from a significantly larger corpus. Hence,
"green", "apple", ?
Assign weights taken at Step 1 to the N-grams of Step 2.
Step 4, computing the weight of the sentence of the training set
It is a bit tricky because you have to invent a good mechanism of computing the weight of entire sentence here. Start with a plain multiplication, but get ready if depending on the nature of your training set, you will have to do a more sophisticated weight computation, for example, using unigrams and/or trigrams as well.
Make sure that you gracefully handle zero values, here's why. Imagine your training set contains a greeting phrase like:
Hello John, welcome to my city
Although it should be a high-rated phrase, "John welcome" may be missing from the corpus hence leading to multiplication to zero. This is called "zero-frequency problem". See this article for more details.
Step 5, getting the final result
Since you have the weights of training sentences, just sort it along with its translation.
I don't think you need to invent your own tools for building N-grams. There are plenty of those, here's just a single link. You may just use them and then load the results into a spreadsheet program and write a small code of final computing and sorting.