how adaptiveWords and baselineWords change their values in Studio 2017 analyses to non-zero

I would need some documentation about how do some 2017 report numbers work in analyses. Read, how do numbers of both "AdaptiveMT Baseline" and "Adaptive MT with Learnings" show up and the process behind. I've tried connecting LC but I see those values again set to zero after my test analysis. As we need to map those numbers to our erp system, I'd like to know how and when they become <> 0 (I assume a simple and informative subset of New/AT? Just like tokens for their reference lines, right?). Is there any specific documentation for that? I've only found what they are basically, but I can't figure out how and the reason behind their change from 0 to something different because this will affect the way the system is picking them up. The xml line I'm referring to is adaptiveWords="0" baselineWords="0". At the end and in a nutshell, I need a way to repro a change of their values.
Best, Matteo Rozzarin

  • Hi Cesare,

    To populate these figures shouldn't be too hard.  Just create an adaptive engine and then analyse your files.  I did this and see this for example:

    I then go through the translation a little, with no TM as I am only working with the AdaptiveMT and after correcting a few here and there I clear the translation completely and then re-analyse with this same adaptive engine (note I only worked on a few segments out of the whole file):

    This time I can see the benefit of using the adaptiveMT as it's taking into account where an improved translation would have been used.

    However, having said this it's still MT and the translation could still be wrong.  In my example some of it will be perfect I hope because I have used it on the same document as if I was doing it from scratch, but some may be where the engine now thinks it can use what it has learned in a different context.  It may well be better, but it may not be.  So this analysis is only a guide to give you some idea of the benefits you may be getting through using the adaptive engines.  If your translators complain that the results are just as bad then it can help you identify texts to test with and see who things could be improved.  So used in conjunction with a tool like Qualitivity for example this can be a very useful piece of intelligence to help improve the engines and the post-editing process.  If you haven't used that tool this article might be wort a read:

    multifarious.filkin.com/.../

    Regards

    Paul

    Paul Filkin | RWS Group

    ________________________
    Design your own training!

    You've done the courses and still need to go a little further, or still not clear? 
    Tell us what you need in our Community Solutions Hub

  • Hi Paul,
    got your point, and could see the numbers shifting from New/AT to the categories below, that was the process I needed to have clear actually. Thanks! Matteo
  • I can't make the AdaptiveMT change to non-zero values... I get only values in "New/AT".
    Does this actually work with the FREE LanguageCloud subscription which does not allow to create adaptive engine, but only has one baseline engine?!

     

    EDIT:

    The reason I need this is to understand the relation between the numbers in various 'categories' (report rows) in order to be able to parse the report XML and extract the analysis results from it (to push it to company project management tool).
    This is of course not documented anywhere... :-\

  • It does work Evzen, but perhaps you are using a non-available language pair? AdaptiveMT is only available for these languages at the moment:

    EN -> DE, ES, FR, IT, NL
    FR -> EN
    IT -> EN
    NL -> EN

    Details of the packages can be found here:

    http://www.sdl.com/software-and-services/translation-software/machine-translation/language-cloud/package-information.html

    Paul Filkin | RWS Group

    ________________________
    Design your own training!

    You've done the courses and still need to go a little further, or still not clear? 
    Tell us what you need in our Community Solutions Hub

  • I'm not that stupid, of course... I'm using the sample project, which contains supported pairs (BTW, how comes that EN->JP does get autotranslated as well if it's not supported?)

    And how comes that Japanese is actually listed?
    Greenshot capture

  • Unknown said:
    I'm not that stupid, of course... I'm using the sample project, which contains supported pairs (BTW, how comes that EN->JP does get autotranslated as well if it's not supported?)

    I don't think you're stupid Evzen, but you do seem to be misunderstanding something.  SDL LanguageCloud does provide MT for EN->JP, but you can't use AdaptiveMT.  AdaptiveMT only comes into play with the supported pairs because you can't add an adaptive engine to any of the others.  So did you create an AdaptiveMT engine and then use it to correct the MT so you have something in there that might be used to provide an improved analysis?

    Paul Filkin | RWS Group

    ________________________
    Design your own training!

    You've done the courses and still need to go a little further, or still not clear? 
    Tell us what you need in our Community Solutions Hub

  • Well, I would not misunderstand if there would be clear documentation somewhere, not just marketing articles/videos. For example the Studio help - where one would expect to get the information in the first place - doesn't say anything helpful :(

    Your terminology is so terribly confusing... I expect that if I use "Baseline" engine, I will see some figures in the "AdaptiveMT Baseline" row.
    And if the engine 'learns' something from updates sent back to it during translation, I would expect to see some figures in the "AdaptiveMT with Learnings".
    That's what simple common sense says... So why is it not like that?!

    Regarding the Japanese - well, I'm just reading the dialog displayed in my Studio... it clearly says AdaptiveMT technology in the bold title and I can also clearly see the "Baseline" word there (which I see also mentioned in the analysis report along with the "AdaptiveMT" word), i.e. I make a pretty clear conclusion that it IS indeed adaptive engine...
    If it is not, then you need to make appropriate changes in the GUI to not confuse users...

    Greenshot capture 

    BTW, the screenshot above has original size 700 x 550 pixels in MS Paint (where I pasted it from GreenShot PNG)... and after copying and pasting from Paint I got this miniature... so I wonder what will be the resulting image after I post this post...

  • Unknown said:
    Your terminology is so terribly confusing... I expect that if I use "Baseline" engine, I will see some figures in the "AdaptiveMT Baseline" row.
    And if the engine 'learns' something from updates sent back to it during translation, I would expect to see some figures in the "AdaptiveMT with Learnings".
    That's what simple common sense says... So why is it not like that?!

    ok, let me see if I can explain this to you.  The first thing is you don't normally get an analysis from MT.  You can pretranslate with MT but not analyse against it.  But if you are using AdaptiveMT then you can analyse against it... at least you can get an indication of how much of the file would be translated against the baseline compared with how much was translated and benefited from your AdaptiveMT engine.  But before you do this you have to actually use an Adaptive Engine.  In your screenshots you are clearly not using any Adaptive engine you are only using the baselines.  If you were then you might see something like this for example:

    Note that I have actually clicked onto the "Manage Adaptive Engines" button and then created two engines, one for FR and one for DE.  I can't create one for JP because this is not supported yet and the language is not available to me in the "Manage Adaptive Engines".

    If I analyse the project now then I see this:

    Note that I am getting results against the AdaptiveMT Baseline.  I get zero against the "learnings" because this is brand new engine I have just created and it has no learnings yet.  It is just a copy of the baseline... completely uncorrected.

    If I was to use it for a while and update the corrections to the AdaptiveMT engine and then run the analysis against this same source file then I would see any segments that were handled using the corrected baseline reported in the "AdaptiveMT with Learnings" row.

    Does that make sense to you?  All it's doing is trying to give you an indication of how much value you are getting from your AdaptiveMT based on the number of corrections you have made over time.  Could be a useful indication when analysing domain specific content and where you have different engines for each domain.  It might help indicate which engine is going to be more suitable to the material you are translating.

    But unless you actually create an AdaptiveMT engine through the "Manage Adaptive Engines" button you won't get a thing!

    Unknown said:
    Regarding the Japanese - well, I'm just reading the dialog displayed in my Studio... it clearly says AdaptiveMT technology in the bold title and I can also clearly see the "Baseline" word there (which I see also mentioned in the analysis report along with the "AdaptiveMT" word), i.e. I make a pretty clear conclusion that it IS indeed adaptive engine...
    If it is not, then you need to make appropriate changes in the GUI to not confuse users...

    I'm reading it now and it says this:

    Machine Translation with AdaptiveMT Technology

    If you don't create an Adaptive Engine then you are not using the AdaptiveMT Technology.  You are just using the baselines.

    Having said all of this Evzen, you are not a stupid guy and yet you have no idea how this works and have completely misunderstood it.  I took a quick look at the help and it does explain what you have to do and shows you how to recognise whether you are using an AdaptiveMT engine or not.  But, as with most of our documentation it is all bare minimum and doesn't really take you by the hand and lead you through it.  It also doesn't explain the analysis part at all and this is far more difficult to understand than how to use the engines in the first place.

    So I totally agree with you that we need to address this.  I'm pretty sure  has this in hand.

    Regards

    Paul

    Paul Filkin | RWS Group

    ________________________
    Design your own training!

    You've done the courses and still need to go a little further, or still not clear? 
    Tell us what you need in our Community Solutions Hub

  • Actually I realized all that right away - I just had to ignore the confusing and counter-intuitive GUI and rely on my own experience.

    It looks like yet another developer-ish UX, basically assuming that one already knows how it all works, instead of GUIDING user to INTUITIVELY DISCOVER the functionality :-\.

    Anyway, what I still don't get is how did you achieve that you have figures in BOTH the "New/AT" AND "AdaptiveMT Baseline"... I can get only EITHER this OR that, not both.
    If I let the sample project pre-translate with the "When no match found: Apply AT" option on, I get wordcount only in "New/AT", but zero in "AdaptiveMT Baseline" (I have the TMs lookup AND the Language Cloud MT provider turned on... and this time I do use the adaptive engine)...
    If I don't let the project pre-translate, then I get wordcount only in the "AdaptiveMT Baseline" row, but zero in "New/AT"...
    So how did you do it?

    Again, the goal of this entire exercise is to get as-'complete'-as-possible analysis report XML, so that I can create versatile parser for Studio 2017 analysis XML which will be robust enough to always extract/calculate correct wordcount from it.