MT quality estimation (MTQE) is an AI-powered feature that provides segment-level quality estimations for machine translation (MT) suggestions. It is similar to the quality estimations for translation memory (TM) matches and non-translatables (NT).
The instant MT quality scores help guide post-editing and can be used to enhance the default (pre-translation) analysis and to assess MT engine quality.
MTQE is only available through the Phrase Translate Add-on.
Using MTQE
MTQE is only available in projects with Phrase Translate as the selected MT engine type.
MTQE does not support all language combinations. See supported language pairs.
Analyses
With MTQE enabled, default analysis includes MT scores alongside the TM and NT scores. For example, a 75% MT score falls into the 75%-84% match range, and a 99% MT score into the 95%-99% match range. This can be turned off in analysis options.
Pre-translation
In addition to the instant, segment-level, quality matches in the CAT editor, MTQE is used in pre-translation. This can be turned off in pre-translate options.
Scoring categories:
-
100% -Excellent MT match, probably no post-editing required
-
99% - Near-perfect MT output, possibly minor post-editing required for mostly typographical errors
-
75% - Good MT match, but likely to require some post-editing
-
No score - When there is no score, it is very likely that the MT output is of low quality. In general, it is recommended that this output not be post-edited but used for reference only.
MTQE scores appear at the segment level together with other translation resources (TM, NT, TB). Match origin is presented in a tooltip and at the bottom of the CAT panel in the metadata section.
Once MTQE has been enabled and employed as part of the machine translation process, MTQE scores for content and engine can be measured for accuracy. Direct comparison between segment-level MTQE scores and post-editing analysis is not available, but the following options provide ways to quantitatively and qualitatively evaluate MTQE scores.
Evaluating with post-editing analysis
Post-editing analysis indicates editing effort; how much text the Linguist or Proofreader had to edit. For post-editing analysis in projects with machine translation and MTQE, results are calculated as the difference between the machine translation suggestions and the final text after post-editing is finished.
In order to evaluate the results of the post-editing analysis, run a default analysis before the first step of the workflow to see how MT matches are categorized into MTQE bands.
When post-editing is complete, run a post-editing analysis with the Analyze MT option.
If the machine translation or non-translatable suggestion was accepted without any editing, the results will indicate 100%.
If the machine translation has been changed, the match rate is lower and the more the segment is changed, the lower the score will be. This is the same score-counting algorithm as the one used to calculate the score of translation memory fuzzy matches.
If the default analysis indicates a high number of quality MT matches (75% or above), the post-editing analysis reflects the correspondingly minimal to moderate amount of editing to the MT suggestions.
Evaluating the segment changes
To evaluate the substance of the changes made during post-editing, create a workflow that generates a report showing the changes on a segment-level.
To create this workflow, follow these steps:
-
Create a project with two sorkflow steps (e.g. pre-translation and post-editing).
-
In the first sorkflow step, pre-translate the job with only MT. This provides a snapshot of the matches to be used.
-
In the second workflow step, let the linguist post-edit normally.
-
Once the workflow is completed, run the post-editing analysis to see the edit distance between the two steps (the number of changes).
-
Select the relevant jobs, then go to and select Export workflow changes.
The different versions of the segments are presented.
Codes are based on ISO 639-1.
|
Source |
Target |
|---|---|
|
ar |
en |
|
ca |
es |
|
cs |
bg |
|
cs |
de |
|
cs |
en |
|
cs |
es |
|
cs |
fr |
|
cs |
hu |
|
cs |
it |
|
cs |
pl |
|
cs |
ro |
|
cs |
ru |
|
cs |
sk |
|
cs |
sl |
|
cs |
sv |
|
cs |
uk |
|
cy |
en |
|
da |
de |
|
da |
en |
|
da |
es |
|
da |
fi |
|
da |
fr |
|
da |
nb |
|
da |
nl |
|
da |
pl |
|
da |
sv |
|
de |
cs |
|
de |
da |
|
de |
en |
|
de |
es |
|
de |
fi |
|
de |
fr |
|
de |
it |
|
de |
lv |
|
de |
nb |
|
de |
nl |
|
de |
pl |
|
de |
ro |
|
de |
ru |
|
de |
sk |
|
de |
sl |
|
de |
sr |
|
de |
sv |
|
el |
en |
|
en |
af |
|
en |
ar |
|
en |
as |
|
en |
az |
|
en |
bg |
|
en |
bn |
|
en |
bs |
|
en |
ca |
|
en |
cs |
|
en |
cy |
|
en |
da |
|
en |
de |
|
en |
el |
|
en |
es |
|
en |
et |
|
en |
fa |
|
en |
fi |
|
en |
fil |
|
en |
fr |
|
en |
ga |
|
en |
gu |
|
en |
he |
|
en |
hi |
|
en |
hr |
|
en |
hu |
|
en |
hy |
|
en |
id |
|
en |
is |
|
en |
it |
|
en |
ja |
|
en |
kk |
|
en |
kn |
|
en |
ko |
|
en |
lt |
|
en |
lv |
|
en |
mi |
|
en |
ml |
|
en |
mr |
|
en |
ms |
|
en |
mt |
|
en |
nb |
|
en |
ne |
|
en |
nl |
|
en |
or |
|
en |
pa |
|
en |
pl |
|
en |
pt |
|
en |
ro |
|
en |
ru |
|
en |
sk |
|
en |
sl |
|
en |
sq |
|
en |
sr |
|
en |
sv |
|
en |
ta |
|
en |
te |
|
en |
th |
|
en |
tl |
|
en |
tr |
|
en |
uk |
|
en |
ur |
|
en |
vi |
|
en |
zh-hans |
|
en |
zh-hant |
|
es |
cs |
|
es |
de |
|
es |
en |
|
es |
fr |
|
es |
it |
|
es |
nl |
|
es |
pt |
|
es |
ru |
|
et |
en |
|
et |
fi |
|
et |
lt |
|
et |
lv |
|
et |
ru |
|
fi |
de |
|
fi |
en |
|
fi |
et |
|
fi |
ru |
|
fi |
sv |
|
fr |
cs |
|
fr |
de |
|
fr |
en |
|
fr |
es |
|
fr |
id |
|
fr |
it |
|
fr |
nl |
|
fr |
pt |
|
fr |
ru |
|
hr |
de |
|
hr |
en |
|
id |
en |
|
it |
cs |
|
it |
de |
|
it |
en |
|
it |
es |
|
it |
fr |
|
it |
pl |
|
it |
pt |
|
it |
ru |
|
ja |
en |
|
ja |
es |
|
ja |
fr |
|
ja |
id |
|
ja |
ko |
|
ja |
ru |
|
ja |
zh-hans |
|
ja |
zh-hant |
|
ko |
en |
|
ko |
ja |
|
lt |
en |
|
lt |
et |
|
lt |
lv |
|
lt |
pl |
|
lt |
ru |
|
lv |
en |
|
lv |
et |
|
lv |
ru |
|
nb |
da |
|
nb |
de |
|
nb |
en |
|
nb |
fi |
|
nb |
nn |
|
nb |
sv |
|
nl |
de |
|
nl |
en |
|
nl |
fr |
|
pl |
cs |
|
pl |
de |
|
pl |
en |
|
pl |
it |
|
pl |
lt |
|
pl |
nl |
|
pl |
ro |
|
pl |
ru |
|
pl |
sk |
|
pt |
en |
|
pt |
es |
|
pt |
fr |
|
pt |
it |
|
ro |
en |
|
ru |
cs |
|
ru |
el |
|
ru |
en |
|
ru |
es |
|
ru |
et |
|
ru |
fr |
|
ru |
ky |
|
ru |
lt |
|
ru |
lv |
|
ru |
pt |
|
sk |
cs |
|
sk |
de |
|
sk |
en |
|
sk |
hu |
|
sl |
en |
|
sr |
en |
|
sv |
ar |
|
sv |
da |
|
sv |
de |
|
sv |
en |
|
sv |
es |
|
sv |
et |
|
sv |
fa |
|
sv |
fi |
|
sv |
fr |
|
sv |
is |
|
sv |
it |
|
sv |
nb |
|
sv |
nl |
|
sv |
pl |
|
sv |
ru |
|
th |
en |
|
tr |
ar |
|
tr |
en |
|
uk |
en |
|
vi |
en |
|
zh-hans |
en |
|
zh-hans |
ja |
|
zh-hant |
en |