Sarah Dussen 

The message seems to be saying "you have TUs in your file that are not nl-BE".  The implications being that your project in Extract is nl-BE.  So if you change all your codes to something other than nl-BE you'll make things worse, not better.

Perhaps you can confirm the languages of your Extract project, and then confirm the languages in your original TMX?

Thank you for your reply Paul.

I'm not sure what you mean? The target language I used for the extract project was nl-BE, given that the tmx target language was also nl-BE.

Hi Paul

Thank you for your detective work!

You are right, the extraction works despite the bug on the small file (207 kb) with 161 TUs.

Hoewever it doesn't work on our actual database of +265.000 KB and +200.000 TUs I'm actually trying to extract from.

The regex you provided works, but only to find just one faulty TU at a time.

Do you know of a regex that could erase the faulty TU's in our big extraction file from <tu to </tu>?

Sarah Dussen 

Sarah Dussen said:

Do you know of a regex that could erase the faulty TU's in our big extraction file from <tu to </tu>?

Do I know of one... no ;-) Can I create one... yes.  But it may not work for all files as it could be dependent on the specific format of your TMX files.  So based on the sample you sent me this seems to work.

Search for this:

<tu[^>]*>(?:(?!<\/tu>).)*(<seg>(?:\s*<(?:bpt|ept|ph)[^>]*\/?>\s*)*<\/seg>|<seg>\s*<\/seg>)(?:(?!<\/tu>).)*<\/tu>

Replace with nothing!

Hm.. We're not quite there yet ...

I tried the string (in Notepad++ I have to add, search mode Regular expression), and 0 results were found.

That shouldn't be possible as it includes the tu of the sample I sent you.

Do you have any more ideas?

Sarah Dussen 

It's often troublesome to use something without fully understanding it!  Make sure you check the "dot matches newline" checkbox... seems to work just fine:

That may be, but I'm determined to find a solution, so willing to learn trying to understand!

I'm getting this error message now:

Sarah Dussen 

And can you paste the regular expression you used in here so we can see it exactly.  Clearly there is nothing wrong with it for Notepad++ or I would have had the same message.  This is what I used:

<tu[^>]*>(?:(?!<\/tu>).)*(<seg>(?:\s*<(?:bpt|ept|ph)[^>]*\/?>\s*)*<\/seg>|<seg>\s*<\/seg>)(?:(?!<\/tu>).)*<\/tu>

Sarah Dussen 

I don't really use Notepad++ but I deliberately broke my expression to see what that little note at the end of the message does and it seems to tell you what's wrong with it if you hover over it:

Maybe share that... or use it to fix your own problem?

Sarah Dussen 

Maybe your Notepad++ version is too old. Latest version is 8.5.6 and fixes are implemented every few days.

Try upgrading from https://notepad-plus-plus.org/downloads/.

Thank you for that suggestion Jesús.

I installed version 8.5.6 only yesterday.

This is the message I'm getting:

Sarah Dussen 

ok - so you must be running this on your larger file?

I think the error is probably due to the complexity of the regular expression, which is taking too long to evaluate.  Regular expressions that have lots of nested repetitions or non-greedy matches can be highly inefficient, especially on large texts.  You may fare better with another editor... I use EditPadPro which handles far larger files than notepad++ can work with, but even with this I don't know how well it would work with your specific file.  You'd need to test and see.

I also don't think I know how to simplify this expression with regex alone as you'd have to start getting too specific and this would remove the benefit of the regex in the first place, so you may be better off using another method to clean up such large files.  Maybe Powershell or Python for example which can make use of some built in libraries that make it easier to manipulate XML.

Sarah Dussen 

ok - so you must be running this on your larger file?

I think the error is probably due to the complexity of the regular expression, which is taking too long to evaluate.  Regular expressions that have lots of nested repetitions or non-greedy matches can be highly inefficient, especially on large texts.  You may fare better with another editor... I use EditPadPro which handles far larger files than notepad++ can work with, but even with this I don't know how well it would work with your specific file.  You'd need to test and see.

I also don't think I know how to simplify this expression with regex alone as you'd have to start getting too specific and this would remove the benefit of the regex in the first place, so you may be better off using another method to clean up such large files.  Maybe Powershell or Python for example which can make use of some built in libraries that make it easier to manipulate XML.