translation memories corrupt

Question

Dear All, 
 Is there a "SDL support" team ? Where / how can i get in touch with them about translation memories that get corrupt without any apparent reason. 
 Thank you 
 angelo russo

Paul Filkin · Answer

Angelo Russo In fact I have just finished recovering what I could. It's a bit of a story so I'll share it for several reasons: it's a good lesson on why you need to backup regularly as this is not a situation you want to be in it's good for others to see how you might be able to recover some TUs from a corrupt TM (it also may not be possible at all if the TM is too far gone) I'm just quite pleased with myself having learned something new ;-) The TM was in fact what we would normally refer to as unrecoverable. If I run a sqlite integrity check on the TM it returns this: *** in database main *** Page 3011: btreeInitPage() returns error code 11 This is not good (google it), and it prevents software from being able to read the contents of the database. So the TM Repair tool we have that is sometimes able to dump the contents to sql and then import it again fails. It just creates an empty TM as the "repaired" version because the contents are irrecoverable. So, I opened the SDLTM in a sqlite browser and could see the tables. So I tried exporting the translation units table into various formats to see what could be done with them. The most readable export was in fact json and this got me a file full of these for each TU: { "align_model_date" : "2020-06-09 17:03:21" , "alignment_data" : "AQABAAwACAEAAIA/BQILAQcBAWAtZD8EAAALAAcBAACAPwUBCAEEAQGAvQE/BAAAAA==" , "change_date" : "2020-08-13 07:14:32" , "change_user" : "FOUND\Angelo" , "creation_date" : "2020-08-13 07:14:18" , "creation_user" : "FOUND\Angelo" , "flags" : 131073 , "guid" : "6l3X31kJYUi6347cFHrGPw==" , "id" : 8247 , "insert_date" : "2020-08-13 07:14:32" , "last_used_date" : "2020-08-13 07:14:32" , "last_used_user" : "FOUND\Angelo" , "source_hash" : 1072023879 , "source_segment" : " Je ne connais pas la personne. nl-BE" , "source_token_data" : "AgwAQQAAAAKsgQKsgQesgQOsgQKsgQik" , "target_hash" : 4527601750 , "target_segment" : " Non conosco la persona. it-IT" , "target_token_data" : "AggAQQAAAAOsgQeswQKsgQek" , "tokenization_sig_hash" : -xxxxxxxxxxxxxxxxxx , "translation_memory_id" : 1 , "usage_counter" : 0 } In here I can at least read the source and target content (bolded). I don't know how large your TM was because I can't open it in Studio at all. Studio tells you it's corrupt, and it also caused me to have to reset the TM View as it corrupted that as well, possibly something else you experienced. But I exported around 7005 TUs. So the problem now is how best to get the text out of a file like this. First of all I tried regex: (?<="source_segment":.+?)[^<]+ (?<="target_segment":.+?)[^<]+ With these expressions I can recover the source into one file and the target into another. This works nicely, but when aligning them there are a lot more target than source so there must be some line breaks or something somewhere in the file and the effort to align them correctly is to much for me (not being a speaker of these languages). So I then had another idea and used the brilliant JSON filetype from Supertext on the appstore and created a filetype that would parse these strings as a bilingual file. I created a few custom rules under the embedded content node to try and exclude tags as it is very taggy but it worked. The general expression for the file was this: It's a great advert for the appstore because I could not do this with the out of the box JSON filetype since it only has monolingual capability. Would be better to see this for 2021 as well through as I had to use 2019 to do this. The embedded rules were things like this... four or five following a similar syntax: Now I can open the file in Studio and as you can see it's still very taggy: But, I created a new TM and updated the contents into it using a batch task. This all worked and now I have a TM... albeit pretty taggy. So then I used the SDLTMConvert app (brilliant tool from Costas Nadalis), also from the appstore, to export the SDLTM as a TMX and clean it to remove the tags along the way. Now I have a clean TMX like this: With this I can now create a new SDLTM. It won't be as good as your original because the context information is lost, formatting tags etc. are lost, but at least you have the plain text which should be better than nothing. I'll send you the SDLTM, TMX, JSON files after I post this. I hope it's useful and perhaps you can use the same approach to do this yourself with your other TMs. It's pretty time consuming and I had to do it between other things happening today... but now we have the process it's not rocket science and might be helpful if your other TMs are too corrupt for the repair tool, but not so corrupted that nothing can be recovered at all.

Paul Filkin · Answer

Angelo Russo 
 Angelo Russo said: How do I recuperate the repaired TM ? 
 I have sent the files back to the email address you used in wetransfer. I put them in dropbox so you should be able to click on the link I provided and download the files. 
 Angelo Russo said: do i need to reinstall STUDIO 2019 and how can I avoid in the future that Onedrive destroys all the rest of my TM ? 
 I'm surprised the support team didn't help you with this, or with your error messages. I knew they wouldn't help with the corrupt TM. However, after using your TM and seeing what it did to my version of 2019 I was able to fix it like this: 
 
 close Trados Studio 
 delete this file c:\Users\[USERNAME]\AppData\Local\SDL\SDL Trados Studio\15.0.0.0\TranslationMemoryRepository.xml 
 Restart Studio and don't use the corrupt TMs 
 
 Give that a try and if it doesn't help then a reinstall might be the best approach, but worth cleaning out all of these folders after the uninstall to make sure it's as clean as possible before reinstalling: 
 1. Uninstall Studio 2. Uninstall MultiTerm 
 Now manually delete these folders if they are still there (back up your TMs, Termbases, AutoSuggest Dictionaries and Projects first in case you make a mistake): 
 c:\ProgramData\Package Cache\SDL c:\Program Files (x86)\SDL\SDL Trados Studio\Studio15\ c:\Program Files (x86)\SDL\SDL MultiTerm\MultiTerm15\ c:\Users\[USERNAME]\AppData\Local\SDL\SDL Trados Studio\15\ c:\Users\[USERNAME]\AppData\Local\SDL\SDL Trados Studio\Studio15\ c:\Users\[USERNAME]\AppData\Local\SDL\SDL MultiTerm\MultiTerm15\ c:\Users\[USERNAME]\AppData\Local\SDL\SDL Trados Studio\15\ c:\Users\[USERNAME]\AppData\Roaming\SDL\SDL Trados Studio\Studio15\ c:\Users\[USERNAME]\AppData\Roaming\SDL\SDL MultiTerm\MultiTerm15\ c:\Users\[USERNAME]\Documents\Studio 2019\Project Templates\ c:\Users\[USERNAME]\Documents\Studio 2019\Projects\projects.xml 
 I think that's about it for a clean install. Then reinstall. 
 Angelo Russo said: how can I avoid in the future that Onedrive destroys all the rest of my TM ? 
 IF you are using OneDrive for these files (it wasn't clear at all for me) then get some help to make sure that you have configured OneDrive to not include the folders where they are. You should also get some advice on how to back up regularly. There are users here who can help you with good advice in this regard, like Jerzy Czopik for example.

Trados Studio > 1. Trados Studio

translation memories corrupt