System Out of Memory error

I regularly have Studio projects containing around 2500 files. Some operations on such projects take > 5 hours (for instance opening an analysis report) or are impossible (for instance opening all files in the Editor, filtering a subset, locking the subset and saving the files again) because Studio hangs or crashes with a System.OutOfMemory error message. It is not the word volume that causes the trouble, but it is the number of files.
The problem can be reproduced at any time.

Conditions: 
Windows 10
Studio 2017 (with latest updates)
Studio 2019 (with latest updates)
intel I7 8th gen, 16 GB RAM, 512 SSD HDD (also tried on lesser equipped machinery)

 
Steps tried but no result:
Unlinked all TM's from project
Project resurrected
RAM usage monitored: never exceeded 3 GB (This is a 32 bit app!).
Removed all SDL temp files from \appdata\local\temp (there are lot left behind after a crash)
Clean start, no programs open but Studio

 

 

 

 

Parents
  • Hi Marcel,

    Here is something you might want to try.

    There is a setting at "File->Options->File Types->SDL XLIFF->General" where you set the "Maximum embedded file size (MB)"

    The default is 20MB, which means that if the source file is smaller than 20MB, then Studio will include the entire source file in the sdlxliff file.

    For example, I created a simple Word file with one sentence and a picture. File size 720KB. With the default setting (20MB), the resulting sdlxliff file was 978KB in size.

    When I changed the above setting to zero, the resulting sdlxliff file was 5KB in size.

    So, big deal, you say. Studio is probably smart enough not to load the entire sdlxliff file into memory. After all, what good would it be to load the embedded source file into memory?

    Well, at least that was what I thought before I did the following test:

    I created 2500 copies of my test file and created two projects, one with the default 20MB setting and one with the setting set to zero.

    When I tried to load the 2500 files that were 978KB in size, Studio very quickly worked its way up to around 2GBM of memory and then threw an out-of-memory exception.

    When I loaded the 2500 files that were 5KB in size, Studio took a bit of time, but never used more than around 0.5GB of memory, and successfully loaded all of the files.

    I guess Studio's memory use is far more inefficient that I previously thought.

    Marcel, if you have a typical project with a large number of files that is easily accessible, you might want to try changing this setting to zero and re-creating the sdlxliff files.

    Then let us know whether this makes a difference with the files you typically use.

    Best regards,
    Bruce Campbell
    ASAP Language Services

  • Hi Bruce,

    I really appreciate it that you took the time to create a test project.

    You found a very interesting track into OOM mystery land. Let me research your findings on my own projects and I'll report asap.

    best
    Marcel Brugmans
Reply Children
No Data