A translator post-edits a file pretranslated with MT. How to calculate total PEM - between MT proposal and the final translator's version - if the translator changes segment more than once?
I can imagine at least two cases:
- Segment was visited two times, once changed, once not. This is trivial - the PEM should be taken from the record where translation was changed.
- Segment visited two times and changed a bit each time. This is non-trivial 
Hello Daniel!
To keep your bike metaphor: it's not about pricing the fixing of a bike, it's about selecting the best wrench
My clients need to calculate PEM not for paying the translators for their effort (I would not recommend that model either), but to choose the best MT for a given language pair and domain. I usually advise to combine 3 factors: PE time, PEM/edit distance and translators' subjective opinion (sometimes also proofreaders' opinion). I wouldn't say it's "my bad" when a translator changes segments >1 time - it is more about the style of work, e.g. getting the meaning right in the first pass and cleaning up the terminology/consistency (usual generic MT flaws) in the second pass. So, I am looking for a method to get the total PEM between MT proposal and the final translator's output, no matter how many times it was edited...?
Post-Edit Compare would be an option but the same client observes an issue similar to this and cannot use PEC in production: community.sdl.com/.../88494
Hello Paul!
Thank you for the considerations. Paying translators per post-editing effort is certainly not a good idea and not my question either, so let's leavethat be (or rather, not be).
But selecting a MT engine is a real-life case. While not all translators may have Qualitivity installed and activated all the time, it does make sense to use it for a limited period of time and on selected "typical" projects, if more than one MT is available for a language pair and domain. It is not too late but just in time to evaluate the first results (3 factors as I wrote above) and decide about further production use.
I know to what level of detail Qualitivity can record data, and I am thinking how to make the best use of it. While post-editing time can be calculated in an easy and reliable way, the PEM has to be "handled with care" as I see it. Btw. in academic research I spotted recently, they just drop all segments visited more than once I am not sure I'd recommend that to my clients, though.
The more I think about it, the more I feel like reaching to a fellow programmer for writing a nice little tool to calculate Levenshtein distance between raw MT proposals and final translators output!
Hello Daniel! Yes, I know the Intento reports and their methodology. In general, Intento's evaluation of stock engines is a good starting point if looking for MT(s). Still, the time may come when we need to figure out which is more productive - e.g. a stock model with grand linguistic quality, or a trained model with perfected terminology
How to implement LEPOR on data gathered from Trados/Qualitivity, would be a separate question, though...
Translate
