Machine Translation

MT Quality Estimation (TMS)

MT quality estimation (MTQE) is an AI-powered feature that provides segment-level quality estimations for machine translation (MT) suggestions. It is similar to the quality estimations for translation memory (TM) matches and non-translatables (NT).

The instant MT quality scores help guide post-editing and can be used to enhance the default (pre-translation) analysis and to assess MT engine quality.

MTQE is only available through the Phrase Translate Add-on.

Using MTQE

MTQE is only available in projects with Phrase Translate as the selected MT engine type.

MTQE does not support all language combinations. See supported language pairs.

Analyses

With MTQE enabled, default analysis includes MT scores alongside the TM and NT scores. For example, a 75% MT score falls into the 75%-84% match range, and a 99% MT score into the 95%-99% match range. This can be turned off in analysis options.

Pre-translation

In addition to the instant, segment-level, quality matches in the CAT editor, MTQE is used in pre-translation. This can be turned off in pre-translate options.

Quality Scores

Scoring categories:

  • 100% -Excellent MT match, probably no post-editing required

  • 99% - Near-perfect MT output, possibly minor post-editing required for mostly typographical errors

  • 75% - Good MT match, but likely to require some post-editing

  • No score - When there is no score, it is very likely that the MT output is of low quality. In general, it is recommended that this output not be post-edited but used for reference only.

MTQE scores appear at the segment level together with other translation resources (TM, NT, TB). Match origin is presented in a tooltip and at the bottom of the CAT panel in the metadata section.

Evaluating MTQE Results

Once MTQE has been enabled and employed as part of the machine translation process, MTQE scores for content and engine can be measured for accuracy. Direct comparison between segment-level MTQE scores and post-editing analysis is not available, but the following options provide ways to quantitatively and qualitatively evaluate MTQE scores.

Evaluating with post-editing analysis

Post-editing analysis indicates editing effort; how much text the Linguist or Proofreader had to edit. For post-editing analysis in projects with machine translation and MTQE, results are calculated as the difference between the machine translation suggestions and the final text after post-editing is finished.

In order to evaluate the results of the post-editing analysis, run a default analysis before the first step of the workflow to see how MT matches are categorized into MTQE bands.

When post-editing is complete, run a post-editing analysis with the Analyze MT option.

If the machine translation or non-translatable suggestion was accepted without any editing, the results will indicate 100%.

If the machine translation has been changed, the match rate is lower and the more the segment is changed, the lower the score will be. This is the same score-counting algorithm as the one used to calculate the score of translation memory fuzzy matches.

If the default analysis indicates a high number of quality MT matches (75% or above), the post-editing analysis reflects the correspondingly minimal to moderate amount of editing to the MT suggestions.

Evaluating the segment changes

To evaluate the substance of the changes made during post-editing, create a workflow that generates a report showing the changes on a segment-level.

To create this workflow, follow these steps:

  1. Create a project with two sorkflow steps (e.g. pre-translation and post-editing).

  2. In the first sorkflow step, pre-translate the job with only MT. This provides a snapshot of the matches to be used.

  3. In the second workflow step, let the linguist post-edit normally.

  4. Once the workflow is completed, run the post-editing analysis to see the edit distance between the two steps (the number of changes).

  5. Select the relevant jobs, then go to Tools and select Export workflow changes.

    The different versions of the segments are presented.

MTQE Supported Language Pairs

Codes are based on ISO 639-1.

Source

Target

ar

en

ca

es

cs

bg

cs

de

cs

en

cs

es

cs

fr

cs

hu

cs

it

cs

pl

cs

ro

cs

ru

cs

sk

cs

sl

cs

sv

cs

uk

cy

en

da

de

da

en

da

es

da

fi

da

fr

da

nb

da

nl

da

pl

da

sv

de

cs

de

da

de

en

de

es

de

fi

de

fr

de

it

de

lv

de

nb

de

nl

de

pl

de

ro

de

ru

de

sk

de

sl

de

sr

de

sv

el

en

en

af

en

ar

en

as

en

az

en

bg

en

bn

en

bs

en

ca

en

cs

en

cy

en

da

en

de

en

el

en

es

en

et

en

fa

en

fi

en

fil

en

fr

en

ga

en

gu

en

he

en

hi

en

hr

en

hu

en

hy

en

id

en

is

en

it

en

ja

en

kk

en

kn

en

ko

en

lt

en

lv

en

mi

en

ml

en

mr

en

ms

en

mt

en

nb

en

ne

en

nl

en

or

en

pa

en

pl

en

pt

en

ro

en

ru

en

sk

en

sl

en

sq

en

sr

en

sv

en

ta

en

te

en

th

en

tl

en

tr

en

uk

en

ur

en

vi

en

zh-hans

en

zh-hant

es

cs

es

de

es

en

es

fr

es

it

es

nl

es

pt

es

ru

et

en

et

fi

et

lt

et

lv

et

ru

fi

de

fi

en

fi

et

fi

ru

fi

sv

fr

cs

fr

de

fr

en

fr

es

fr

id

fr

it

fr

nl

fr

pt

fr

ru

hr

de

hr

en

id

en

it

cs

it

de

it

en

it

es

it

fr

it

pl

it

pt

it

ru

ja

en

ja

es

ja

fr

ja

id

ja

ko

ja

ru

ja

zh-hans

ja

zh-hant

ko

en

ko

ja

lt

en

lt

et

lt

lv

lt

pl

lt

ru

lv

en

lv

et

lv

ru

nb

da

nb

de

nb

en

nb

fi

nb

nn

nb

sv

nl

de

nl

en

nl

fr

pl

cs

pl

de

pl

en

pl

it

pl

lt

pl

nl

pl

ro

pl

ru

pl

sk

pt

en

pt

es

pt

fr

pt

it

ro

en

ru

cs

ru

el

ru

en

ru

es

ru

et

ru

fr

ru

ky

ru

lt

ru

lv

ru

pt

sk

cs

sk

de

sk

en

sk

hu

sl

en

sr

en

sv

ar

sv

da

sv

de

sv

en

sv

es

sv

et

sv

fa

sv

fi

sv

fr

sv

is

sv

it

sv

nb

sv

nl

sv

pl

sv

ru

th

en

tr

ar

tr

en

uk

en

vi

en

zh-hans

en

zh-hans

ja

zh-hant

en

Was this article helpful?

Sorry about that! In what way was it not helpful?

The article didn’t address my problem.
I couldn’t understand the article.
The feature doesn’t do what I need.
Other reason.

Note that feedback is provided anonymously so we aren't able to reply to questions.
If you'd like to ask a question, submit a request to our Support team.
Thank you for your feedback.