Sarah Dussen 

The message seems to be saying "you have TUs in your file that are not nl-BE".  The implications being that your project in Extract is nl-BE.  So if you change all your codes to something other than nl-BE you'll make things worse, not better.

Perhaps you can confirm the languages of your Extract project, and then confirm the languages in your original TMX?

Thank you for your reply Paul.

I'm not sure what you mean? The target language I used for the extract project was nl-BE, given that the tmx target language was also nl-BE.

 Sarah Dussen 

ok -so read what you have written again.  I can only go on what you're telling us.

Sarah Dussen said:

"Warning! There were translation units (tu) found in file (name of my TMX file) which did not have a translation unit variant (tuv) with target language "Dutch (Belgium)"

It says it found TUs in your file that did not have nl-BE language codes.

You then said this:

Sarah Dussen said:

I was able to change all target tuvs from nl-BE to nl, but that did not help. Changing them to nl-NL did not help either.

Why?  If the TMX contained TUs that were not nl-BE then you have made certain there aren't any at all.

Or of course I'm completely misunderstanding what you have written here?

Maybe you can share your TMX so I can try it?

I realise it's contradictory, but it was the solution that was suggested in the only Q&A I found, to the same problem I had, just with a different target language.

I would be grateful if you could give it a try. What would be a safe way to share my tmx to you privately?

Sarah Dussen 

Sure... pfilkin at sdl dot com

Thank you! I sent you the export yesterday by e-mail.

Sarah Dussen 

Thanks for the file... found some time to look at it this evening.  I get the same error you mentioned, but the application still extracts the terminology:

So, I split the file and kept splitting it until I found the problem.  It relates to the segment contents rather than the language codes so this was a misleading error message.  The segment (<seg>) for both the fr-BE and nl-BE languages contains placeholders (<ph>, <bpt>, and <ept>) but no actual translatable content in one single <tu>. The placeholders refer to certain tag formats or structures that were present in the original document.  If you're extracting terminology, the actual terms or sentences should be inside the <seg> tags and since this file you provided has a single TU with only placeholders and no content Extract seems to get a bit upset!

This is the offending translation unit variant:

      <tuv xml:lang="fr-BE">
        <seg><bpt i="1" type="110" x="1" /><ph x="2" type="114" /><ept i="1" /></seg>
      </tuv>
      <tuv xml:lang="nl-BE">
        <seg><bpt i="1" type="110" x="1" /><ph x="2" type="114" /><ept i="1" /></seg>
      </tuv>

If you search for that and delete the entire translation unit and then try again the error is not there.

Sarah Dussen 

Thanks for the file... found some time to look at it this evening.  I get the same error you mentioned, but the application still extracts the terminology:

So, I split the file and kept splitting it until I found the problem.  It relates to the segment contents rather than the language codes so this was a misleading error message.  The segment (<seg>) for both the fr-BE and nl-BE languages contains placeholders (<ph>, <bpt>, and <ept>) but no actual translatable content in one single <tu>. The placeholders refer to certain tag formats or structures that were present in the original document.  If you're extracting terminology, the actual terms or sentences should be inside the <seg> tags and since this file you provided has a single TU with only placeholders and no content Extract seems to get a bit upset!

This is the offending translation unit variant:

      <tuv xml:lang="fr-BE">
        <seg><bpt i="1" type="110" x="1" /><ph x="2" type="114" /><ept i="1" /></seg>
      </tuv>
      <tuv xml:lang="nl-BE">
        <seg><bpt i="1" type="110" x="1" /><ph x="2" type="114" /><ept i="1" /></seg>
      </tuv>

If you search for that and delete the entire translation unit and then try again the error is not there.

Sarah Dussen 

Just to close this off... you can use this regex to quickly find segments like this... also segments with no content at all... and that should save some time trying to find the offending segments:

<seg>(?:\s*<(bpt|ept|ph)[^>]*\/?>\s*)*<\/seg>

Hi Paul

Thank you for your detective work!

You are right, the extraction works despite the bug on the small file (207 kb) with 161 TUs.

Hoewever it doesn't work on our actual database of +265.000 KB and +200.000 TUs I'm actually trying to extract from.

The regex you provided works, but only to find just one faulty TU at a time.

Do you know of a regex that could erase the faulty TU's in our big extraction file from <tu to </tu>?

Sarah Dussen 

Sarah Dussen said:

Do you know of a regex that could erase the faulty TU's in our big extraction file from <tu to </tu>?

Do I know of one... no ;-) Can I create one... yes.  But it may not work for all files as it could be dependent on the specific format of your TMX files.  So based on the sample you sent me this seems to work.

Search for this:

<tu[^>]*>(?:(?!<\/tu>).)*(<seg>(?:\s*<(?:bpt|ept|ph)[^>]*\/?>\s*)*<\/seg>|<seg>\s*<\/seg>)(?:(?!<\/tu>).)*<\/tu>

Replace with nothing!

Hm.. We're not quite there yet ...

I tried the string (in Notepad++ I have to add, search mode Regular expression), and 0 results were found.

That shouldn't be possible as it includes the tu of the sample I sent you.

Do you have any more ideas?

Sarah Dussen 

It's often troublesome to use something without fully understanding it!  Make sure you check the "dot matches newline" checkbox... seems to work just fine:

That may be, but I'm determined to find a solution, so willing to learn trying to understand!

I'm getting this error message now:

Sarah Dussen 

And can you paste the regular expression you used in here so we can see it exactly.  Clearly there is nothing wrong with it for Notepad++ or I would have had the same message.  This is what I used:

<tu[^>]*>(?:(?!<\/tu>).)*(<seg>(?:\s*<(?:bpt|ept|ph)[^>]*\/?>\s*)*<\/seg>|<seg>\s*<\/seg>)(?:(?!<\/tu>).)*<\/tu>

Sarah Dussen 

I don't really use Notepad++ but I deliberately broke my expression to see what that little note at the end of the message does and it seems to tell you what's wrong with it if you hover over it:

Maybe share that... or use it to fix your own problem?

Sarah Dussen 

Maybe your Notepad++ version is too old. Latest version is 8.5.6 and fixes are implemented every few days.

Try upgrading from https://notepad-plus-plus.org/downloads/.

Thank you for that suggestion Jesús.

I installed version 8.5.6 only yesterday.

This is the message I'm getting: