RunAutomaticTask with AnalyzeFiles seems to run very slow when number of ProjectFileIds input is large

Have any one ever come across an issue when running a RunAutomaticTask with AnalyzeFiles API with large number of files in the project?

Here's my comparison:

This one had less than 100 files in the project:

This one had about ten times more files:

I would expect 10 times longer, but 37 minutes is definitely much longer than 5 minutes.

Have anyone ever come across this issue? Is there any way to solve this?

Thank you!

Rieko

Parents
  • Analysis is quite a complex process.

    Here is an example of an analysis I ran today:

    >I would expect 10 times longer, but 37 minutes is definitely much longer than 5 minutes

    Unfortunately it isn't that simple. A couple things to consider:

    1. Are all the files the exact same size and have the same exact structure? Unless that is the case, each file will take a different amount of time to analyze.

    2. Each file is located at a different location on your disk. The time it takes to find and open each one varies greatly.

    3. Trados (and PCs in general) have a limited amount of resources, once you use them up, extra work has to be done.

    For example, with a few files there maybe no need to release any other memory, but once you increase the number of files

    you process, then extra work has to be done to release memory which didn't occur when you had only a few files.

    Anyways, one thing you can do is try upgrading your hardware, i.e. increasing RAM, getting a faster CPU or a solid-state drive.

    However, 783 files in 37 minutes seems quite fast so maybe you already have a fast PC.

    The other solution is to wait and see if the SDL developers can improve how analysis is performed, but I wouldn't count on that.

Reply
  • Analysis is quite a complex process.

    Here is an example of an analysis I ran today:

    >I would expect 10 times longer, but 37 minutes is definitely much longer than 5 minutes

    Unfortunately it isn't that simple. A couple things to consider:

    1. Are all the files the exact same size and have the same exact structure? Unless that is the case, each file will take a different amount of time to analyze.

    2. Each file is located at a different location on your disk. The time it takes to find and open each one varies greatly.

    3. Trados (and PCs in general) have a limited amount of resources, once you use them up, extra work has to be done.

    For example, with a few files there maybe no need to release any other memory, but once you increase the number of files

    you process, then extra work has to be done to release memory which didn't occur when you had only a few files.

    Anyways, one thing you can do is try upgrading your hardware, i.e. increasing RAM, getting a faster CPU or a solid-state drive.

    However, 783 files in 37 minutes seems quite fast so maybe you already have a fast PC.

    The other solution is to wait and see if the SDL developers can improve how analysis is performed, but I wouldn't count on that.

Children