Exporting a Server TM using the TranslationMemory API results in a TMX file that misses the context matching information

Hi, originally, I posted this question here, but I think I chose the wrong forum at least there was zero feedback ... So I post here again and hope to receive feedback here Slight smile

I have detected a problem with the Exporter class in the TranslationMemory API. The problem is that - when a server TM gets exported, the TMX is missing the context information after the export.

Here is my code:

Dim TMExporter As New Sdl.LanguagePlatform.TranslationMemoryApi.TranslationMemoryExporter(ServerBasedTranslationMemory.GetLanguageDirection(LangDir))

TMExporter.ChunkSize = 100

TMExporter.FilterExpression = GetFilterCreatedAt()

AddHandler TMExporter.BatchExported, AddressOf Exporter_BatchExported

strExportfile = ExportFolder & "\" & ServerTM & "_" & LanguageDirection.SourceLanguageCode.ToString & "_" & LanguageDirection.TargetLanguageCode.ToString & ".tmx"

Try

      Dim strNow As String = Now.ToString("yyyyMMdd_HHmmss")

      TMExporter.Export(strExportfile, True)

      System.IO.File.AppendAllText(LogFile, ControlChars.CrLf & strNow & " SUCCESS: " & ServerTM & " , Source Language: " & LanguageDirection.SourceLanguageCode.ToString & " , Target Language: " & LanguageDirection.TargetLanguageCode.ToString
      & " was successfully exported to " & strExportfile & ". " & exportProgress.ToString & " TUs were exported." & ControlChars.CrLf)

Catch ex As Exception

      Dim strNow As String = Now.ToString("yyyyMMdd_HHmmss")

      System.IO.File.AppendAllText(LogFile, ControlChars.CrLf & strNow & " ERROR: " & ServerTM & " , Source Language: " & LanguageDirection.SourceLanguageCode.ToString & " , Target Language: " & LanguageDirection.TargetLanguageCode.ToString &
      " could not be exported. The original error message was: " & ex.ToString & ControlChars.CrLf)

      System.IO.File.Delete(ExportFolder & "\" & ServerTM & "_" & LanguageDirection.SourceLanguageCode.ToString & "_" & LanguageDirection.TargetLanguageCode.ToString & ".tmx")

End Try

This code - when run on a local SDLTM exports everything just fine including the context information:

Screenshot of Trados Studio TMX code with highlighted section showing missing context information after export from a server TM.

But when I run it on a server TM, the TMX misses the context information after the export finishes:

Screenshot of Trados Studio TMX code with no visible errors or warnings, context information appears to be present.

I can also export the server TM from Trados Studio and from the Groupshare Web UI just fine and the exports always include the context information.

Also, as I said before - export from a SDLTM through the API also creates a TMX that includes the context information. 

It is really only the export from a server TM through the TM API that loses the context information. Is this a known issue? It basically means that - whereever automation of exporting TMX from server TMs is used to backup the TM data, this will currently lose the context information. 



Generated Image Alt-Text
[edited by: Trados AI at 4:12 AM (GMT 0) on 5 Mar 2024]
Parents
  • Hi  , I have reviewed this issue today, but unable to reproduce the issue that you are reporting.

    Prerequisites

    • Trados Studio 2022 SR1 - 17.1.6.16252
    • GS: 15.1.9.12529 - GS 2020 SR1 CU09

    Steps

    1. Create a server based tm from GroupShare
    2. Open the server based TM in Stduio & add some custom fields
    3. Add a few TU entries & assign a custom field value to one or two of the TU's
    4. Commit changes so that they are updated on the server.
    5. Create a standalone app to connect and export the server based TM to tmx
      1. Documentation: How to export serverbased TM to TMX

    Actual result

    The custom field values are exported correctly, visible from the TU's in the TMX that was exported from the serverbased TM

    Follow up

      Can you pls provide the GroupShare version + Trados Studio version you are using?

  • Dear Patrick, 

    many thanks for your answer. My issue is about the *context matching information* - not about custom fields. Custom fields - I agree - do survive the export. The context matching information in the TM is stored in these fields:

    <prop-type="x-IncludesContextContent">True</prop>

    This turns the feature on, I guess.

    Then on segment pair level, you will find:

    <prop type="x-Context"> etc. as highlighted in the screenshot:

    Screenshot of Trados Studio XML code with highlighted properties including 'x-IncludesContextContent' set to 'True' and 'x-Context' with segment pair level information.

    These elements - when you export via the API from a server TM - are *not* exported. I am quite sure you can repro it. I tested against an older GS 2020 SR 1 CU7 (gsdemo.gs.sdl.com) as well as my own VM running GS 2020 SR1 8.1. I will test against CU9 as well, but I am quite sure it is not a Groupshare issue - it is a Studio API issue. 



    Generated Image Alt-Text
    [edited by: Trados AI at 4:12 AM (GMT 0) on 5 Mar 2024]
  • Hi Patrick,

    ok, I will test with CU9 then and report back - it would be wonderful if it was fixed already in CU9 Slight smile 

  • Hi Patrick,

    my test with CU9 was negative - with CU9, this code causes an internal server error (500) - it is still working fine against CU7 and CU8.1 of Groupshare (running Trados Studio 2022 SR1):

    >>>>>>>>>>

    Public Function GetServerTMs()

    Dim ServerUri As New System.Uri(ServerURL & ":" & Portnumber)

    Dim TMServer As New TranslationProviderServer(ServerUri, False, Username, Password)

    For Each ServerBasedTranslationMemory In TMServer.GetTranslationMemories().   <---- Internal server Error (500) happens here 

    Dim ServerTM As String = ServerBasedTranslationMemory.Name

    Msgbox(ServerTM)

    Next

    Return 0

    End Function

    >>>>>>>>>>>

    However, I will create a separate request for this issue, as this has (I think) nothing to do with my original ticket here.

    But fact is - with CU9, I cannot retrieve any TMs any longer using the code above. This code as worked for years without any issues. Disappointed

  • Hi   - there is indeed a known issue in the API when retrieving the list of GroupShare TMs, however you can work around it in your code by making certain small changes. I would ask   to explain the workaround here when he has time. Also tagging  . Thanks, Daniel

    Daniel Brockmann
    Team Trados @ RWS

  • Hi  , I have just received information from  that you should use  "GetTranslationMemoriesByQuery(TranslationMemoryEntityQuery query)" to get the TMs + avoid this exception. 

    Can you try this an let me know how it goes?

  • Hi Patrick,

    I tried, but the result is exactly the same as with the GetTranslationMemories function.

    Screenshot of Trados Studio error message displaying 'System.AggregateException: One or more errors occurred.' and 'System.Net.Http.HttpRequestException: Response status code does not indicate success: 500 (Internal Server Error).'

    The error in the GS TMService.log is also identical:

    2023-08-25 08:24:50.2662|WIN-DGSP368NVJB|Error|THREAD_ID:78|TR_ID:|EX:Object reference not set to an instance of an object.|DefaultExceptionHandler System.NullReferenceException: Object reference not set to an instance of an object.
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.ParsingHelper.GetPropertyName(CollectionNavigationNode property)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.AnyNodeParser.ParseInternal(AnyNode expression, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.AbstractSingleValueNodeParser`1.Parse(SingleValueNode node, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.ODataQueryOptionsParser.Parse(SingleValueNode node, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.ODataQueryOptionsParser.Parse[T](SingleValueNode node, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.UnaryOperatorNodeParser.ParseInternal(UnaryOperatorNode expression, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.AbstractSingleValueNodeParser`1.Parse(SingleValueNode node, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.ODataQueryOptionsParser.Parse(SingleValueNode node, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.BinaryOperatorNodeParser.ParseInternal(BinaryOperatorNode expression, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.AbstractSingleValueNodeParser`1.Parse(SingleValueNode node, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.ODataQueryOptionsParser.Parse(SingleValueNode node, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.BinaryOperatorNodeParser.ParseInternal(BinaryOperatorNode expression, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.AbstractSingleValueNodeParser`1.Parse(SingleValueNode node, String resourceIdFieldName)
    at Sdl.Services.Common.ResourceQuery.WebApi.Parsing.ODataQueryOptionsParser.Parse(SingleValueNode node, String resourceIdFieldName)
    at Sdl.Services.Common.AspNetCore.RestApi.OData.ODataQueryOptionsExtensions.ToResourceFilterExpression(SingleValueNode odataFilterExpression, String resourceIdFieldName)
    at Sdl.Services.Common.AspNetCore.RestApi.OData.ODataQueryOptionsExtensions.ToResourceListOptions[T](ODataQueryOptions`1 options, Int32 maxNodeCount)
    at Sdl.TMService.RestApi.Controllers.TranslationMemoriesController.Get(ODataResourceQueryOptions`1 options) in D:\a\1\s\TMService\RestApi\Controllers\TranslationMemoriesController.cs:line 1224
    at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
    --- End of stack trace from previous location ---
    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
    at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
    at Sdl.TMService.Common.Runtime.EntitlementVerificationMiddleware.InvokeAsync(HttpContext context) in D:\a\1\s\TMService\Sdl.TMService.Common.Runtime\EntitlementVerificationMiddleware.cs:line 46
    at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
    at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
    at Sdl.Services.Common.AspNetCore.RestApi.Middleware.ExecutionContextMiddleware.InvokeAsync(HttpContext context)
    at Sdl.Services.Common.AspNetCore.RestApi.Authentication.AuthenticationRequestMiddleware.InvokeAsync(HttpContext context)
    at Sdl.TMService.Common.Runtime.ExceptionHandling.ExceptionHandlingMiddleware.Invoke(HttpContext context) in D:\a\1\s\TMService\Sdl.TMService.Common.Runtime\ExceptionHandling\ExceptionHandlingMiddleware.cs:line 30



    Generated Image Alt-Text
    [edited by: Trados AI at 4:13 AM (GMT 0) on 5 Mar 2024]
  • Hello  

    Can you please please tell us, please, what parameters you send in the TranslationMemoryQuery.

    Also, here is an example of parameters that works: 

    var input = new TranslationMemoryEntityQuery()
    {
    Size = 10,
    Index = 1,
    IsProject = true,
    OwnerId = GuidOwnerId,
    IncludeChildResourceGroups = true,
    ResourceGroupPath = "Location"
    };

    Also send paratmeter IsProject as true, so you'll get all TMs.

    Can you, please, try this an let us know how it goes?

  • Hello  

    How TranslationMemoryQuery looks?

    Here you have an working example: 

    var query = new Sdl.LanguagePlatform.TranslationMemoryApi.TranslationMemoryQuery() {
    ResourceGroupPath = UserSettings.Organization,
    IsProject = true,
    IsMain = true,
    IncludeChildResourceGroups = false
    };

    Also, set IsProject as true.

    Can you, please, try this an let us know how it goes?

    Thanks!

  • Hello Paul,

    many thanks, I will try this and get back to you on the results. 

    Kind regards

    Tom

  • Hi Patrick,

    In the meantime I have tested the following two use cases using an older GS-Server

    Running export using (classic) TranslationMemoryExporter:

    --> Original problem (context information missing in the TMX) can be reproduced.

    versus:

    Running export with the new method ScheduledServerTranslationMemoryExport:

    --> Original problem cannot be reproduced.

    So, for the original problem, I have a solution at least and can carry on with my work :-) Many thanks for the decisive tipp about using the scheduled export! However, the issue should also not occur using the TranslationMemoryExporter method, even if it is not recommended as it is slower than the scheduled one.

    Many thanks,

    Tom

  • Hi  , thank you for confirming this (y). I confirm that I can also reproduce the original issue you reported against GS server CU9 using TranslationMemoryExporter.  I overlooked this as I've always used ScheduledServerTranslationMemoryExport  for server based memories as it is the more optimal approach, especially over slower network.

    You're absolutely correct, the content of the exported TMX should be the same irrespective of the users choice in using the scheduled export or not.   ,  I'll escalate this issue to the dev team.

Reply Children