• State Not Answered
  • Replies 6 replies
  • Subscribers 3 subscribers
  • Views 2770 views
  • Users 0 members are here
Options
Related

TM Batch Export: Studio API: Context Match information are missing

Frantisek Spicak

 Hi,

we have "TM Batch Exporter", it is custom development. It exports TMs from TM server according to certain criteria into TMX files. It uses for Export TranslationMemoryExporter Class. Simply put:

- The "TM Batch Exporter" works in combination Studio Client 2017 up to latest CU + GS 2015. All information are there, specifically context match information.

- It does not work with same version of Studio Client + GS 2017 CU8, the exported files do not contain context match information which are very important, without those the export is good for nothing. 

Note: If we use REST API or export server based TM via client manually all context info. are exported.

From the SDK documentation is clear there is not and has never been API parameter that specifies either include or exclude the context match info. So this has changed on GS side without properly upgrading API on Client side to reflect this change. 

We already logged SDL case but it was resolved with suggestion to ask here in dev. forum. 

 

Do you anyone face similar issue? 

thanks and regards,

Frantisek
Rate translation
×
Suggest better translation
×
Parents
  • 0
    Hi Frantisek

    We are investigating the issue and will hopefully resolve it soon.

    In the meantime, you could change your exporter to use the same method as the Studio 2017/2019 applications. The TranslationMemoryExporter implementation downloads a batch of TUs and writes them out to the export file. It then gets the next batch of TUs... and then the next... and then the next... and so on. This is not very efficient for exporting GroupShare based TMs. Studio 2017 requests the export to be performed on the server, and then the exported file is compressed and downloaded. It is normally much quicker.

    1. First queue the export on GroupShare 2017:

    var scheduledExport = new ScheduledServerTranslationMemoryExport(sbLanguageDirection);
    scheduledExport.FilterExpression = anyfilter;
    scheduledExport.DownloadCancelEvent = downloadCancelEvent;
    scheduledExport.Queue();

    2. Wait for the export on the server to complete:

    scheduledExport.Refresh();

    while (scheduledExport.Status != ScheduledOperationStatus.Aborted
    && scheduledExport.Status != ScheduledOperationStatus.Cancelled
    && scheduledExport.Status != ScheduledOperationStatus.Completed)
    {
    // Give the server a chance to do something.
    Thread.Sleep(4000);

    scheduledExport.Refresh();

    ReportProgress((int)((scheduledExport.TranslationUnitsProcessed * 100.0 / TotalTuCount) * 75.0 / 100.0),
    string.Format("Waiting... {0} TUs exported", scheduledExport.TranslationUnitsExported)));
    }

    3. Download the export file and decompress:

    // Currently all server-based TMs are downloaded in a compressed format (.tmx.gz). If a TMX file is required
    // then the extension passed to the server must be amended to end in .tmx.gz
    // Once downloaded, the TMX file must be decompressed.
    string fileToDownloadTo = _exportFilePath;
    bool unzipRequired = scheduledExport.IsStreamCompressed;
    if (unzipRequired)
    {
    fileToDownloadTo = fileToDownloadTo + ".gz";
    }

    using (Stream destinationStream = File.Open(fileToDownloadTo, FileMode.Create))
    {
    scheduledExport.DownloadExport(destinationStream);
    }

    if (unzipRequired)
    {
    using (FileStream zipFile = File.OpenRead(fileToDownloadTo))
    using (GZipStream decompressor = new GZipStream(zipFile, CompressionMode.Decompress))
    using (FileStream tmxFile = File.Create(exportFilePath))
    {
    decompressor.CopyTo(tmxFile);
    }

    File.Delete(fileToDownloadTo);
    }

    Don't expect this code to compile without changes but it should give an idea how we export GroupShare TMs in Studio 2017/2019

    Best regards,

    Iain.
  • 0 in reply to Iain Shaw

    Hello Iain, thank you for your prompt reply. We had actually tried that one as well. The export is triggered without problems. But The line scheduledExport.DownloadExport just hangs forever, even with a TM that contains only 10 TUs. I let it run for 30 minutes, then had to abort, because all I got was an empty TMX- oder GZ file.
  • 0 in reply to Ziad Chama
    Hi Ziad,
    That's strange. I've tried that code this afternoon with a GroupShare 2017 CU07 server and a CU08 server (with Studio 2017 SR1 CU14) and the exported file was downloaded as expected. Did you try downloading the file from the GroupShare Web UI in the Background Tasks (System Configuration) for the same export task?

    With GroupShare 2017, the 'DownloadExport' method is simply a wrapper for a REST API call. E.g. GET myGroupshareServer/.../output

    Iain.
  • 0 in reply to Iain Shaw
    Hello Iain, I can indeed download the TMX from the Web UI of the CU8 development server. But my sample application still gets stuck when downloading the same GZ file programmatically. It creates a GZ file with 0 KB and just hangs there at the line that is applying the DownloadExport method.

    using (Stream destinationStream = File.Open(fileToDownloadTo, FileMode.Create))
    {
    exporter.DownloadExport(destinationStream);
    }
  • 0 in reply to Ziad Chama
    Hi Ziad,
    Do you have an app like Fiddler installed. If so, look for the 'output' REST API call. It should have a URL like this:

    '/api/tm-service/v1/tasks/<import-task-guid>/output'.

    Is there an error code? or is the call timing out?
    You could try looking for any errors in the TMService.log (%ProgramData%\SDL\Service\Logs) on the server at the time of the DownloadExport call.
Reply Children