MT in the translation ecosystemGábor Bessenyei
You justify your investment by the return on costs. However, on the long run, your savings should come from streamlining and not from pressurizing the members of your ecosystem who depend on you.
Straining the members of your organization may work for a while, but in the long run you will lose. You will lose your best translators, editors, proofreaders and create havoc for your project managers. Of course, eventually this again will result in quality problems — problems that will finally make your customers unhappy.
With Globalese, you can integrate machine translation into your workflow in a way that keeps your whole ecosystem satisfied. Your company will save costs and shorten delivery time. Your customers will benefit from reduced turnaround times, better prices and may even ensure higher quality. More excitingly: your translators will become more productive, so their balance will be the same at the end of the day. Yes, they may even find that they like machine translation!
When the development of Globalese started a couple of years ago, there was already a lot of hype generated around machine translation. One of the reasons why we decided to bring a new solution to the market was that we found that despite of the fact that machine translation can really help save time and money, there was no clear answer and solution to the following important questions:
- How can you measure the benefit of using MT and convert it into cents?
- How can you build a process which is fair and acceptable to your whole ecosystem?
The MT scope
Machine-translating segments locked by the project manager, or segments which are already covered by your TMs make no sense: the only result is wasted computing time. Filling the documents with bad machine translations will just waste the translators’ time: all you do is make them read and delete low-quality MT output. Both cases will increase unhappiness with MT and generate unnecessary costs.
So before finding out how to measure, we have to define what to measure. In other words: which segments should be machine-translated in a translation project?
The Globalese team identified the following criteria:
- Segments that are not locked by the project manager
- Segments not covered by Translation Memory perfect or fuzzy matches (75–100%)
- Segments that can be machine-translated by the engine at an acceptable quality level
Measurement is a major headache when adopting MT. The industry has come up with various solutions. Let’s just take a look at the most common market solutions for measuring MT quality and post-editing effort!
- The most common way is the time-based method. In this solution post-editors are asked to report the time spent on their work, which is then compared to benchmark values. The problems are clear: this approach is very subjective, you need a very deep level of trust to rely on the results, and even the benchmark values can differ from project to project. This is why many companies use MT only in projects performed by internal employees.
- The second method is typically used by large language service providers: they simply ask their vendors or freelancers for a fix discount or a discount based on a machine-translated sample. The problem is that there is no guarantee that the quality of the sample relates to the quality of the entire project, presenting a high risk factor to vendors once they have accepted this method for settlement.
- The third method is to compare the distance between the machine translation output and the final post-edited version, segment by segment. The result is a match rate table based on the target language. The advantage here is that you get an exact and objective measurement of the post-editing effort. Unfortunately, it has one disadvantage that hinders wider acceptance: it can be applied only after finalizing the translation.
With Globalese we have developed a new method in addition to the distance analysis of MT output and post-edited version. Globalese will allow you to assess the quality of the MT engine and – get segment by segment values already before project start! The fact that costs can be calculated before the translation project starts can potentially increase the acceptance of MT among customers and translators.
The Globalese way
Globalese works with the bilingual CAT tool formats, and the engines translate only untranslated segments. This means that you can pre-translate your project with the fuzzy and perfect TM hits, and Globalese will only translate the rest.
With the Quality estimation feature you can get a clear picture of your costs and efforts on the segment level before the project starts. This highly increases the acceptance of MT among your customers and freelancers.
By setting a Quality threshold, you can exclude low-quality MT output, so your translators will not waste time with reading and deleting useless MT output.
Once your project is finished, you can optionally pass the finalized translation back to Globalese and use the Evaluation feature to verify your translators as well as the engine quality.
Converting MT into cents
The Quality Estimation feature provides a segment-by-segment report on MT quality. The result is a match rate table where you can apply your discount rates.
Remember: we apply MT only to those segments which are not covered by your translation memories. This means that the quote and the PO will contain two parts: one standard TM analysis for the segments covered by the TMs, and Globalese quality estimation / evaluation analysis for the MT part, showing where the savings are.
The volume of savings depends on the quality of your engine and the word price in the particular project. A realistic expectation is a saving between 10%-40% of your project costs.
The bottom line
Cutting costs and making the members of your ecosystem satisfied is not an impossible mission. If you have a solid and fair way to calculate your savings, and if you are willing to share a portion of it with the members of your ecosystem, you can become not only one of the winners of a win-win situation, but that one that sets the rules.