Idea Delivered

Have you looked into apps for this, such as Qualitivity - https://appstore.sdl.com/language/app/qualitivity/612/ ?

Metrics to quantify the benefit of MT

There are various reasons why someone would want to quantify the benefit of MT in a scenario where it's an additional "provider" in a Studio project.  We all feel that it's a productivity booster, but we just don't know how much.   We also don't know objectively which MT provider gives the best boost in productivity to the translators.

 

 

I would like to see a solution from SDL to solve this in Studio.

For example, something that would track the time it took the translator to post-edit an MT match vs a TM match, vs translating from scratch. So essentially calculate the average translation speed for a manual translation, vs a fuzzy, vs a postedited MT translation.

Also, something that would track how often the MT suggestion is useful, vs discarded and the translator decides to rewrite the translation from scratch.

Also, track how many TUs in our TMs are the results of postedited MT  vs postedited fuzzy matches? 

There are a lot of metrics I could imagine to see the benefit of MT and we can’t be the only one having those types of questions.

Michel Farhi-Chevillard

  • Hi Michel,

    Regarding your first question on a tool that tracks the time it took the translator to post-edit an MT match vs a TM match, have you looked at the Studio plugin Qualitivity for this? Qualitivity records all timestamps for every edit per segment, and data from this tool can easily show this type of correlation. Check out from 10:15 in Paul's final video in this blog entry for more detail:

    multifarious.filkin.com/.../

    Regarding your 2nd question on data regarding how often the MT suggestion is useful vs. discarded, the edit-distance metrics in Qualitivity as well as Post-Edit Compare would give you detail to this effect. For example, any segment that is 50% or higher TERp score in Post-Edit Compare means that half of the words in the segment have been edited, namely almost fully discarded.

    Unfortunately, to your final question regarding types of TU-origins, I am unfamiliar of this type of data from TMs so can't speak to this point.

    Luke