• State Suggested Answer
  • Replies 24 replies
  • Answers 3 answers
  • Subscribers 432 subscribers
  • Views 13538 views
  • Users 0 members are here
Options
Related

System Out of Memory error

Marcel Brugmans

I regularly have Studio projects containing around 2500 files. Some operations on such projects take > 5 hours (for instance opening an analysis report) or are impossible (for instance opening all files in the Editor, filtering a subset, locking the subset and saving the files again) because Studio hangs or crashes with a System.OutOfMemory error message. It is not the word volume that causes the trouble, but it is the number of files.
The problem can be reproduced at any time.

Conditions: 
Windows 10
Studio 2017 (with latest updates)
Studio 2019 (with latest updates)
intel I7 8th gen, 16 GB RAM, 512 SSD HDD (also tried on lesser equipped machinery)

 
Steps tried but no result:
Unlinked all TM's from project
Project resurrected
RAM usage monitored: never exceeded 3 GB (This is a 32 bit app!).
Removed all SDL temp files from \appdata\local\temp (there are lot left behind after a crash)
Clean start, no programs open but Studio

 

 

 

 
Rate translation
×
Suggest better translation
×
Moderator UI
×
  • Get AI Suggestion
Parents Reply Children
  • 0 in reply to Paul
    My point was that the "internal under-the-hood processing complexity" of the resulting SDLXLIFF files is what matters here... i.e. some XML with embedded formatting-rich HTML content (approx. hundred segments each) or complex IDML files, or similar are way more likely to expose the problem than some simple files.
  • 0 in reply to Evzen Polenka
    I understatnd that Evzen which is why I asked for the project. I have an interested developer since I shared this thread so if you have your project I'd be really pleased to be able to share that as well.

    Paul Filkin | RWS Group

    ________________________
    Design your own training!
    You've done the courses and still need to go a little further, or still not clear? 
    Tell us what you need in our Community Solutions Hub

  • 0 in reply to Paul

    From your post, Paul, it sounded like 2500 docx files might be a problem. So I tried it.

    I prepared a small docx file with an image and a small Excel table, and copied it to get around 2500 files.

    I then created a new project and added the files. It took 30 minutes to prepare the project (analyze only).

    I then selected all the files and tried to open them for translation.

    After a few minutes, the Studio process was using around 1.7 GB of memory. Then the following error message was displayed "There is an error in XML document (1,76)".

    The error stack indicates an "out of memory exception".

    I have attached the test file0028.test.docx and the error stack

    Fullscreen sdlerror-20181221-16h0m9s.sdlerror.xml Download 
    <SDLErrorDetails time="21/12/2018 16:00:17">
  <ErrorMessage>There is an error in XML document (1, 76).</ErrorMessage>
  <Exception>
    <Type>System.InvalidOperationException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</Type>
    <HelpLink />
    <Source>System.Xml</Source>
    <HResult>-2146233079</HResult>
    <StackTrace><![CDATA[   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   at Sdl.FileTypeSupport.Bilingual.SdlXliff.XliffFileReader.OnInternalFile(XmlElement internalFile)
   at Sdl.FileTypeSupport.Bilingual.SdlXliff.SdlXliffFeeder.<ContinueScanning>b__14_13(ISdlXliffStreamContentHandler handler)
   at System.Collections.Generic.List`1.ForEach(Action`1 action)
   at Sdl.FileTypeSupport.Bilingual.SdlXliff.SdlXliffFeeder.ContinueScanning()
   at Sdl.FileTypeSupport.Bilingual.SdlXliff.XliffFileReader.ContinueParsing()
   at Sdl.FileTypeSupport.Bilingual.SdlXliff.XliffFileReader.ParseNext()
   at Sdl.FileTypeSupport.Framework.Integration.FileExtractor.ParseNext()
   at Sdl.FileTypeSupport.Framework.Integration.MultiFileConverter.ParseNext()
   at Sdl.FileTypeSupport.Framework.Integration.MultiFileConverter.Parse()
   at Sdl.TranslationStudio.Editor.TranslationEditor.TranslatableDocument.Load(IJobExecutionContext context)
   at Sdl.Desktop.Platform.Services.JobRequest.Execute(IJobExecutionContext context)
   at Sdl.Desktop.Platform.Implementation.Services.Job.<_worker_DoWork>b__46_0()
   at Sdl.Desktop.Platform.Implementation.Services.Log.Resources(Object message, Action action)
   at Sdl.Desktop.Platform.Implementation.Services.Job._worker_DoWork(Object sender, DoWorkEventArgs e)
   at System.ComponentModel.BackgroundWorker.OnDoWork(DoWorkEventArgs e)
   at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)]]></StackTrace>
    <InnerException>
      <Type>System.OutOfMemoryException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</Type>
      <HelpLink />
      <Source>mscorlib</Source>
      <HResult>-2147024882</HResult>
      <StackTrace><![CDATA[   at System.Text.StringBuilder.set_Length(Int32 value)
   at System.Xml.XmlTextReaderImpl.ParseText()
   at System.Xml.XmlTextReaderImpl.ParseElementContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at Sdl.FileTypeSupport.Bilingual.SdlXliff.WhitespacePreservingXmlTextReader.Read()
   at System.Xml.XmlReader.ReadStartElement(String name)
   at Oasis.Xliff12.internalfile.ReadXml(XmlReader reader)
   at System.Xml.Serialization.XmlSerializationReader.ReadSerializable(IXmlSerializable serializable, Boolean wrappedAny)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read63_internalfile()]]></StackTrace>
    </InnerException>
  </Exception>
  <Environment>
    <ProductName>SDL Trados Studio</ProductName>
    <ProductVersion>14.0.0.0</ProductVersion>
    <EntryAssemblyFileVersion>14.1.10015.44945</EntryAssemblyFileVersion>
    <OperatingSystem>Microsoft Windows 10 Pro</OperatingSystem>
    <ServicePack>NULL</ServicePack>
    <OperatingSystemLanguage>1033</OperatingSystemLanguage>
    <CodePage>1252</CodePage>
    <LoggedOnUser>STAR\asaplang</LoggedOnUser>
    <DotNetFrameWork>4.0.30319.42000</DotNetFrameWork>
    <ComputerName>STAR</ComputerName>
    <ConnectedToNetwork>True</ConnectedToNetwork>
    <PhysicalMemory>33417884 MB</PhysicalMemory>
  </Environment>
</SDLErrorDetails>
    .

    I was hoping to be able to open the files to see how long it took to select a subset of the segments, but unfortunately did not get that far...

    Judging from your post, I think you knew this would happen. So what kind of file is needed in order to handle 2500 files in a project? Anything but docx?

    Best regards,
    Bruce Campbell
    ASAP Language Services
  • 0 in reply to Bruce Campbell
    I tried opening around half of the 2500 files and the Studio process got up to around 2GB of memory and then threw the following error: "Exception of type 'System.OutOfMemoryException' was thrown."
  • 0 in reply to Bruce Campbell
    I selected around 1/3 of the files (around 800?, by eyeballing the scroll bar) and was able to open them. Studio was using around 1.8GB of memory. I was able to easily select a sub-set of segments using "^This is a test" in the Display Filter and change their status and lock them.

    I guess if the files are simple, then you can manipulate them easily if Studio is actually able to load them...
  • 0 in reply to Bruce Campbell
    As I pointed out earlier, the memory fragmentation plays a big role here... i.e. you might(!) be able to open the files after a fresh start of Windows (when the memory is not yet fragmented). Also a clean Windows installation will behave differently than a "common" one with quite a number of programs installed, where a number of processes start (and then close, like various checks for updates - Java, Chrome, Adobe, etc.) after start.
  • 0 in reply to Paul
    Paul,

    The projects I have those OOM troubles with are XML and DITAMAP files belonging to a help-system.
    File sizes vary from 1 MB to 5 KB.
    Bilingual SDLXLIFF files are created by the customer.

    best
    //Marcel
  • 0 in reply to Evzen Polenka
    Hi Evzen,

    I guess you are correct that memory fragmentation plays a role. But I had no luck with clean starts or killing as many other apps as possible.The way Windows and Studio treat memory is a bit of a black art.

    An additional complication is that Intel, Microsoft and computer manufacturers restricted the way applications can use memory since the Meltdown and Spectre security vulnerabilities surfaced around a year ago. Which apps suffer most, or not at all, is another black art.
  • 0 in reply to Marcel Brugmans

    Unknown said:
    But I had no luck with clean starts or killing as many other apps as possible.

    Well, killing the apps is actually what creates the memory fragmentation. The point is to prevent the apps from even being started.