This article explains how confident Phrase is in the quality of its translation capabilities for different language pairs, and how to check quality on translated content.
It applies to Phrase NextMT, Phrase Next GenMT, AI Translation Agent, Auto Adapt, and MT Optimize. Each capability has a separate language pair list, but all capabilities use the same confidence tiers.
Why Quality Varies Between Language Pairs
Translation quality depends on many factors, including the amount of training data available for a language. Quality can also differ by translation direction. A pair can perform well from English into a language but less consistently in the opposite direction. The type of content also affects quality.
Third-party machine translation services have their own language lists. Check the languages that the service supports.
Important
The confidence tiers are based on tests using general-purpose content. They provide guidance rather than a guarantee of quality, and Phrase may update them as testing continues. Quality for the same language pair can differ significantly between content types. Check the output on specific content before using any capability at scale.
|
Tier |
What it means |
Suggestion |
|---|---|---|
|
Confident |
The pair was tested in that direction and performed well in Phrase’s tests. |
Use the capability as usual, with standard quality checks. |
|
Limited confidence |
The pair was tested, but the results were not reliable enough to confirm quality. Quality may be inconsistent. Review before use. |
Test a sample of your content and review the output before use. Consider adding post-editing. |
|
Not tested |
The pair has not been tested. Phrase cannot confirm quality. |
Try the capability on a sample of the content and review the output before use. |
Each capability has its own list. The tier applies to each translation direction separately. Language pairs not listed for a capability are Not tested for that capability.
|
Language |
English to language |
Language to English |
|---|---|---|
|
Arabic |
Confident |
Confident |
|
Bulgarian |
Limited confidence |
Not tested |
|
Chinese (Simplified) |
Confident |
Confident |
|
Chinese (Traditional) |
Confident |
Confident |
|
Czech |
Confident |
Confident |
|
Danish |
Confident |
Limited confidence |
|
Dutch |
Confident |
Confident |
|
Finnish |
Confident |
Confident |
|
French |
Confident |
Limited confidence |
|
German |
Confident |
Confident |
|
Greek |
Confident |
Limited confidence |
|
Hebrew |
Limited confidence |
Limited confidence |
|
Hindi |
Confident |
Not tested |
|
Hungarian |
Confident |
Limited confidence |
|
Icelandic |
Confident |
Not tested |
|
Indonesian |
Confident |
Confident |
|
Italian |
Confident |
Limited confidence |
|
Japanese |
Confident |
Confident |
|
Korean |
Confident |
Confident |
|
Malay |
Confident |
Not tested |
|
Norwegian |
Confident |
Confident |
|
Polish |
Confident |
Limited confidence |
|
Portuguese |
Confident |
Confident |
|
Romanian |
Confident |
Not tested |
|
Russian |
Confident |
Limited confidence |
|
Serbian |
Confident |
Confident |
|
Serbian (Cyrillic) |
Confident |
Limited confidence |
|
Serbian (Latin) |
Confident |
Confident |
|
Slovak |
Confident |
Limited confidence |
|
Spanish |
Confident |
Confident |
|
Swedish |
Confident |
Confident |
|
Thai |
Confident |
Confident |
|
Turkish |
Confident |
Confident |
|
Ukrainian |
Confident |
Not tested |
|
Vietnamese |
Confident |
Confident |
|
Welsh |
Confident |
Confident |
Pairs without English
|
Language pair |
Tier |
|---|---|
|
Catalan to Spanish |
Limited confidence |
|
Chinese (Simplified) to Japanese |
Limited confidence |
|
Chinese (Simplified) to Korean |
Limited confidence |
|
Chinese (Simplified) to Thai |
Confident |
|
Chinese (Simplified) to Vietnamese |
Limited confidence |
|
Estonian to Finnish |
Confident |
|
Japanese to Chinese (Simplified) |
Limited confidence |
|
Korean to Chinese (Simplified) |
Confident |
|
Lithuanian to Estonian |
Limited confidence |
|
Lithuanian to Latvian |
Confident |
|
Lithuanian to Russian |
Limited confidence |
|
Spanish to Catalan |
Limited confidence |
|
Thai to Chinese (Simplified) |
Confident |
|
Vietnamese to Chinese (Simplified) |
Confident |
Test NextMT on a sample of content before using it at scale, especially for Limited confidence and Not tested pairs. Include each content type planned for translation, such as UI strings, marketing copy, and support articles.
To evaluate quality, follow these steps:
-
Enable Phrase NextMT on the Phrase Language AI page or, for accounts that use MT profiles, in an MT profile.
Note
NextMT is not enabled by default for existing customers. It is enabled by default for new subscribers and trial users. Only TMS Administrators and Project Managers with permissions to modify global server settings can edit MT profiles.
To make sure the test uses only NextMT, disable all other engines or create a dedicated MT profile that contains only NextMT. If multiple engines are enabled, MT Autoselect picks the engine for each job and may not choose NextMT.
-
Translate the sample in the Phrase product in use. If MT profiles are used, apply the profile that contains only NextMT.
Product
Translate the sample
View the Quality Performance Score (QPS)
Phrase TMS
Assign the profile to a project and pre-translate it.
Enable Display 0-100% Quality Performance Score (QPS) on translations in in the project settings. Scores appear at the segment level.
Phrase Strings
Assign the profile to a language in the project’s language settings and machine translate the sample keys.
Scores appear in the editor when a translation is saved. Toggle the display on and off as needed.
Phrase Portal
Upload a sample file and click Translate, using a portal set up to use NextMT.
Enable QPS for file translations in the of the portal.
Send the sample using the ID of the profile.
The API returns document-level scores.
-
Review the Quality Performance Score (QPS).
QPS predicts the score a human reviewer would give the translation, from 0 to 100. QPS scoring is available by default, but some products require switching on the display, as listed in the table above.
Score
Category
95–100
Excellent
90–94
Very good
75–89
Good
0–74
Poor
Note
QPS is not available for every language pair. Refer to Phrase QPS Overview for the supported pairs.
-
Have a linguist review the sample. If the results meet requirements, continue and keep monitoring quality, especially when the content type changes. Otherwise, add a post-editing step to the workflow.
In Phrase TMS, use Phrase Orchestrator to send only segments below a chosen QPS score for human review. The in Phrase Analytics helps identify a suitable threshold.