• State Not Answered
  • Replies 6 replies
  • Subscribers 3 subscribers
  • Views 2769 views
  • Users 0 members are here
Options
Related

TM Batch Export: Studio API: Context Match information are missing

Frantisek Spicak

 Hi,

we have "TM Batch Exporter", it is custom development. It exports TMs from TM server according to certain criteria into TMX files. It uses for Export TranslationMemoryExporter Class. Simply put:

- The "TM Batch Exporter" works in combination Studio Client 2017 up to latest CU + GS 2015. All information are there, specifically context match information.

- It does not work with same version of Studio Client + GS 2017 CU8, the exported files do not contain context match information which are very important, without those the export is good for nothing. 

Note: If we use REST API or export server based TM via client manually all context info. are exported.

From the SDK documentation is clear there is not and has never been API parameter that specifies either include or exclude the context match info. So this has changed on GS side without properly upgrading API on Client side to reflect this change. 

We already logged SDL case but it was resolved with suggestion to ask here in dev. forum. 

 

Do you anyone face similar issue? 

thanks and regards,

Frantisek
Rate translation
×
Suggest better translation
×
Parents
  • 0
    Hi Frantisek

    We are investigating the issue and will hopefully resolve it soon.

    In the meantime, you could change your exporter to use the same method as the Studio 2017/2019 applications. The TranslationMemoryExporter implementation downloads a batch of TUs and writes them out to the export file. It then gets the next batch of TUs... and then the next... and then the next... and so on. This is not very efficient for exporting GroupShare based TMs. Studio 2017 requests the export to be performed on the server, and then the exported file is compressed and downloaded. It is normally much quicker.

    1. First queue the export on GroupShare 2017:

    var scheduledExport = new ScheduledServerTranslationMemoryExport(sbLanguageDirection);
    scheduledExport.FilterExpression = anyfilter;
    scheduledExport.DownloadCancelEvent = downloadCancelEvent;
    scheduledExport.Queue();

    2. Wait for the export on the server to complete:

    scheduledExport.Refresh();

    while (scheduledExport.Status != ScheduledOperationStatus.Aborted
    && scheduledExport.Status != ScheduledOperationStatus.Cancelled
    && scheduledExport.Status != ScheduledOperationStatus.Completed)
    {
    // Give the server a chance to do something.
    Thread.Sleep(4000);

    scheduledExport.Refresh();

    ReportProgress((int)((scheduledExport.TranslationUnitsProcessed * 100.0 / TotalTuCount) * 75.0 / 100.0),
    string.Format("Waiting... {0} TUs exported", scheduledExport.TranslationUnitsExported)));
    }

    3. Download the export file and decompress:

    // Currently all server-based TMs are downloaded in a compressed format (.tmx.gz). If a TMX file is required
    // then the extension passed to the server must be amended to end in .tmx.gz
    // Once downloaded, the TMX file must be decompressed.
    string fileToDownloadTo = _exportFilePath;
    bool unzipRequired = scheduledExport.IsStreamCompressed;
    if (unzipRequired)
    {
    fileToDownloadTo = fileToDownloadTo + ".gz";
    }

    using (Stream destinationStream = File.Open(fileToDownloadTo, FileMode.Create))
    {
    scheduledExport.DownloadExport(destinationStream);
    }

    if (unzipRequired)
    {
    using (FileStream zipFile = File.OpenRead(fileToDownloadTo))
    using (GZipStream decompressor = new GZipStream(zipFile, CompressionMode.Decompress))
    using (FileStream tmxFile = File.Create(exportFilePath))
    {
    decompressor.CopyTo(tmxFile);
    }

    File.Delete(fileToDownloadTo);
    }

    Don't expect this code to compile without changes but it should give an idea how we export GroupShare TMs in Studio 2017/2019

    Best regards,

    Iain.
Reply
  • 0
    Hi Frantisek

    We are investigating the issue and will hopefully resolve it soon.

    In the meantime, you could change your exporter to use the same method as the Studio 2017/2019 applications. The TranslationMemoryExporter implementation downloads a batch of TUs and writes them out to the export file. It then gets the next batch of TUs... and then the next... and then the next... and so on. This is not very efficient for exporting GroupShare based TMs. Studio 2017 requests the export to be performed on the server, and then the exported file is compressed and downloaded. It is normally much quicker.

    1. First queue the export on GroupShare 2017:

    var scheduledExport = new ScheduledServerTranslationMemoryExport(sbLanguageDirection);
    scheduledExport.FilterExpression = anyfilter;
    scheduledExport.DownloadCancelEvent = downloadCancelEvent;
    scheduledExport.Queue();

    2. Wait for the export on the server to complete:

    scheduledExport.Refresh();

    while (scheduledExport.Status != ScheduledOperationStatus.Aborted
    && scheduledExport.Status != ScheduledOperationStatus.Cancelled
    && scheduledExport.Status != ScheduledOperationStatus.Completed)
    {
    // Give the server a chance to do something.
    Thread.Sleep(4000);

    scheduledExport.Refresh();

    ReportProgress((int)((scheduledExport.TranslationUnitsProcessed * 100.0 / TotalTuCount) * 75.0 / 100.0),
    string.Format("Waiting... {0} TUs exported", scheduledExport.TranslationUnitsExported)));
    }

    3. Download the export file and decompress:

    // Currently all server-based TMs are downloaded in a compressed format (.tmx.gz). If a TMX file is required
    // then the extension passed to the server must be amended to end in .tmx.gz
    // Once downloaded, the TMX file must be decompressed.
    string fileToDownloadTo = _exportFilePath;
    bool unzipRequired = scheduledExport.IsStreamCompressed;
    if (unzipRequired)
    {
    fileToDownloadTo = fileToDownloadTo + ".gz";
    }

    using (Stream destinationStream = File.Open(fileToDownloadTo, FileMode.Create))
    {
    scheduledExport.DownloadExport(destinationStream);
    }

    if (unzipRequired)
    {
    using (FileStream zipFile = File.OpenRead(fileToDownloadTo))
    using (GZipStream decompressor = new GZipStream(zipFile, CompressionMode.Decompress))
    using (FileStream tmxFile = File.Create(exportFilePath))
    {
    decompressor.CopyTo(tmxFile);
    }

    File.Delete(fileToDownloadTo);
    }

    Don't expect this code to compile without changes but it should give an idea how we export GroupShare TMs in Studio 2017/2019

    Best regards,

    Iain.
Children