Hi Marta Bartnicka,

I have downloaded and taken a look at the excel files that you uploaded.  I don’t see any issues with the data that is reported given the content in the reports.

There should be no difference in the behavior of the Post Edit Modification (PEM)% that is calculated, whether you are pre-translating via a batch task or recovering the translations one at a time from a translation provider in the Studio editor.

The Post Edit Modification (PEM)% that is calculated depends on whether or not you had a pre-existing translation before you modified the segment/s.

If you translate a document that had previously been translated and translations exist for the segments that you are updating, then the PEM% will calculate the comparable differences.  If you didn’t have a pre-existing translation before you interactively updated the translation, then the PEM% is zero; except when you are recovering the translations from provider and don't perform any adaption to that translation.

The PEM% is set to 100% for translations that are automatically recovered from translation providers, to suggest: no additional work was understaken by the translator in providing that translation.  Whereas, a PEM% of Zero indicates a new translation was provided interactively in the editor (whether or not that was recovered initially by a provider).

The key point to remember here is.  The Post Edit Modification % is calculated against pre-existing translations in the document from the time you start recording the actions with Qualitivity (normally when you open the document) to the time you stop working on it (normally when you close the document).

Hi Patrick Andrew Hartnett!

Thank you for prompt reply I think I now understand better how this works.

So, if I prefer to enter MT interactively segment by segment, how can I measure the % of my modifications against the MT proposals - for the segments that are not PEM 100% (unmodified)?

Regards,

Marta

Hi Marta Bartnicka,

The translations would need to exist before you start the Qualitivity session, to calculate the modifications analysis against the newer updated translations.

I would recommend to simply perform a preranslation batch task using your translation provider, so that the translations already exist prior to opening the document in the editor.  Then, Qualitivity will perform the PEM% and calculate the comparable differences for any modifications that you apply to the existing translations.

 Patrick Andrew Hartnett, thank you - this is clear now!

I have no further issues with the function and I know how to use it.

If, at some point, you decide to extend Qualitivity functionality by comparing post-edited text to raw MT input inserted interactively, please count me as beta tester

Good suggestion and I remember that this 'type of' question was in discussion during the original beta phases a few years ago.  We eventually decided against it as it can happen that many changes could occur to the translation from the time the document is opened and closed that would make it difficult to understand from which point do perform the comparison (from -> ...) that would be reliable and not cause further ambiguity.

Example

• Translator enters segment 1
• Attempt to translate interactively given the MT suggestion.
• Accepts MT suggestion from the provider instead.
• Modifies the MT translation in the target area
• Changes the TM provider settings so that another TM suggestion is provided; possibly from another provider.
• Accepts the new translation from the provider
• Adapts the translation again... etc..
• Confirms translation for segment 1

It is difficult to identify a reliable basis as the starting point for comparison in this case; other than when the user simply started the task (i.e. opened the document and started the Qualitivity session)

Might be possible to include an option use the first translation provided as the basis for comparison, but that would then provide an incorrect assumption that the translation existed prior to opening the document and interfere with other expectations.

Certainly something to think about though... thanks for your input; I'll keep this in mind.

Patrick.

Patrick Andrew Hartnett, thank you for the details and please excuse me for being silent for a while. I don't have much to add, except for a question, perhaps: How would you recommend to run a comparison of 2 MTs? As I'd see it, one can attach 2 MTs to a translation task (NOT pre-translate) and then compare:

1) From which MT the proposals were chosen more often?

2) Which MT was closer to the final translators' work (=edit distance)?

3) Which MT allowed to save more time?

I'd know how to measure 2) and 3) with Qualitivity on two sets of pre-translated files, each with different MT, and given to two different translators - but that's not quite the same experiment...

I am looking forward to receiving your suggestions.

Regards,

Marta

Marta Bartnicka

I haven’t tested this, but I seem to recall that the excel export of the raw data includes the origin of the MT. If this is correct you could create your report using excel to find the engine that provided the most efficiency.

Paul Thank you, this is partially correct: the "Original Origin System" shows where the proposal came from... but ONLY for a project pre-translated with MT. The column is empty for a project where MT (or 2 MTs) is used dynamically as source of proposals. You can observe that effect in the two raw data repors I included in the original question.

If the "Original Origin System" column is filled with whatever engine was initially chosen by translator for a segment, then I can get all I need from the raw data.

Regards,

Marta

Paul Thank you, this is partially correct: the "Original Origin System" shows where the proposal came from... but ONLY for a project pre-translated with MT. The column is empty for a project where MT (or 2 MTs) is used dynamically as source of proposals. You can observe that effect in the two raw data repors I included in the original question.

If the "Original Origin System" column is filled with whatever engine was initially chosen by translator for a segment, then I can get all I need from the raw data.

Regards,

Marta

Marta Bartnicka

You might be able to get this information by including the keystroke data in the report.  I confirmed with the developer that it's not possible to get what you're after the way you're doing it because the translation origin is is updated to reflect the current state.

Paul

Thank you very much! If the development ever considers adding "Original Origin System" to the report for dynamically used MT, please consider me as beta tester

Regards,

Marta