Scan batch task on Trados API randomly fails with message "The file exists."

Hello, 

I have a typical function to add files to a project. I have a very large catalog of html files that I need to convert, around 70k files. I have set up multiple Python workers for this task and they invoke the C# code with the relevant parameters like source, target lang, output path, input file path. However, this typical code here randomly fails:

private void AddFiles(FileBasedProject project, string sourceFilePath)
{
// AddFiles expects an array of strings
string[] sourceFilePaths = { sourceFilePath };
project.AddFiles(sourceFilePaths);

ProjectFile[] projectFiles = project.GetSourceLanguageFiles();

AutomaticTask scan = project.RunAutomaticTask(projectFiles.GetIds(), AutomaticTaskTemplateIds.Scan);

}

the problem is that the scan object has the message "The file exists.". This Add files function sits here in a typical sequence of calls for Trados automation:

newProject = new FileBasedProject(this.GetProjectInfo(srcLocale, trgLocale, outputpath));
var settings_ = newProject.GetSettings();
var xliffSettings = settings_.GetSettingsGroup("SDL XLIFF 1.0 v 1.0.0.0");
xliffSettings.GetSetting<bool>("ValidateXliff").Value = false;
newProject.UpdateSettings(settings_);

this.AddFiles(newProject, sourceFilePath);

What does this "The file exists." even mean? Can you see where this could come from?

If I shut the process off and then try only a failed task on its own, then it works. Can someone help me on this? I have never encountered this before. I am using Trados 2022.