Machine translation quality prediction

Machine translation quality prediction (“MTQP” or “QP”) is the AI task of predicting if a machine translation is good or bad.

Concretely, the goal is to predict whether the machine-translated segment would be edited or not, if sent to a human translator in that workflow.

In the research world, machine translation quality prediction is also known as “machine translation quality estimation” or “QE.”

The input is a source sentence and its translation, and the output is a score.

Example

INPUT

“The quick brown fox jumps over the lazy dog.”

“Der kecke braune Fuchs gumpt über der lausigen Dogge.”

OUTPUT

→ 0%

INPUT

“The quick brown fox jumps over the lazy dog.”

“Der schnelle braune Fuchs springt über den faulen Hund.”

OUTPUT

→ 90%

How is quality prediction different from a quality metric like BLEU? BLEU can only be used to score old content, in aggregate, because BLEU requires human reference translation, and is not accurate at the segment level - it doesn’t even consider the source text!

Quality prediction doesn't just measure old work, it accelerates new work! Unlike BLEU, it can be applied to new content, in production.

Hybrid translation: blending AI and human intelligence

An introduction to the hybrid translation paradigm