Hi Maciej

Anything in the log file that might cast some light on this?

Daniel

Good morning Daniel,

thank you for your response. 

Yep, there is something, which I paste here below:
4162.errors_4_community.xml
It is my hope that you will figure out something helpful out of this. Finger crossed.
Rgds
Maciej

Maciej Śpiewak

That would be interesting... you can send me a link at pfilkin@sdl.com

As agreed, you may have already gotten a message from WeTransfer.

Download link
https://we.tl/t-vpM2BOPCIn

As a follow-up, you might also want to have the source file...
Once you have solved the puzzle, I shall be delighted, as both the TXT and the TMX are a wealth of reliable usable segments. An effort of 10+ years to gather them.

Download link
https://we.tl/t-zkGRPG568j

Maciej Śpiewak

The problem seems to be illegal chars in eleven of the TUs in the TMX... for example:

I removed the offending TUs completely and it all imported correctly. Studio carries out some validation on the file prior to importing, and will fail with errors in the XML (as above), and will remove TUs that are duplicates, missing source or target etc.  So the final result is this

I sent you a link to where you can download the repaired files.

Paul

Paul,

courtesy of your skills and committment the problem has been fixed, which I haste to confirm here.
So I am deeply grateful. 
Regards
Maciej

Paul,

If I knew of your procedure of fixing that you apply, it would permit me NOT to bother you again, because I have found another one with a great number of TUs and you realise the challenge of finding illegal chars...

Use any editor able to validate XML, for example Notepad++. Or Okapi Olifant - it should also detect such problems. There are also other dedicated TMX editors, just ask uncle Google :)

BTW, the illegal characters are usually the same in the complete TM, so using Notepad++ is a good idea, as you can use decent search & replace function there.

Jerzy, thanks for your feedback, no I need a set of chars that are illegal in order to search and replace them

Install Notepad++, then install the XML additions there.

Open the TMX in Notepad++ and run the XML validation. N++ should bring you to the first "illegal" place. Copy the offending character and paste it into the search box, then replace all occurrences by a "legal" character. Validate again - as long no errors will be reported.

Be careful with that... the illegal char was the use of & on its own.  If you search replace it you'll break any entity in the file (and there are many).  I'm not familiar with the Notepad++ extension so perhaps it's smarter than that... I use sublime text and the Exalt addin which finds problems as soon as I open a file... no need to do anything.  It takes a while with your file as there were over a million lines, but it allowed me to find the problems and correct them.

Be careful with that... the illegal char was the use of & on its own.  If you search replace it you'll break any entity in the file (and there are many).  I'm not familiar with the Notepad++ extension so perhaps it's smarter than that... I use sublime text and the Exalt addin which finds problems as soon as I open a file... no need to do anything.  It takes a while with your file as there were over a million lines, but it allowed me to find the problems and correct them.

Paul, so may you please identify your tool so that I could download it from the net?

Aha, it is the Sublime Text editor, ok.

Paul apparently I managd somehow to install both the Sublime txt and the Exalt addin. And now what? How do I validate/rectify my file?

Maciej Śpiewak

I wasn't expecting you use that... but it is a good editor, you should learn about the basic features.

On using it, you can create a small simple XML example like this and you'll see that it is validated by flagging the problems automatically like this:

You can also see the locations at the bottom like this:

(click on these images to be able to read them)

Hope that helps?

I am at a loss. I have a TMX file not an XML file... I just woul dlike to know a step by step process that you applied to fix my previous bug..

And on top of this, I first heard of Sublime from you today. I have no literacy, no skill and getting disheartened slowly

Maciej Śpiewak said:

I have a TMX file not an XML file...

TMX is XML... look at the header:

Maciej Śpiewak said:

And on top of this, I first heard of Sublime from you today. I have no literacy, no skill and getting disheartened slowly

Basically the problem is that Wordfast is exporting invalid XML so you have some segments in your TMX that you either need to remove, or fix.  These are the invalid chars in XML:

In your file I removed the TUs instead of replacing the & with & because they didn't look as though they should be there anyway and the TUs I removed were actually same source same target, missing source and only target, or just gibberish (not sentences at all)... so removing them was faster.  To do that you need to understand the structure of a TMX.  It's pretty simple, each TU has this structure.. I marked each one with a red square:

You can remove a complete TU (everything in the square) without damaging the file.  So my process was this:

1. Identify which TUs had the invalid segments in them one at a time
2. Decide whether it was worth attempting to replace the invalid char with the correct entity, or some other character, or not
3. Replace the char, or delete the entire TU (what's in the red box)

That's it.  Sometimes you can do smart replacements and perhaps the one from Jerzy with Notepad++ is clever enough to let you replace only the ampersands that should not be there, I don't know.  But that's the approach I took.

I imagine others have their own way too... Evzen Polenka will almost certainly have a smart way of tackling these issues.

But one thing is for sure... if you want to be able to handle things like this you need to read up a little on why it's necessary (google invalid XML), read up on how XML is structured (https://www.w3schools.com/xml/), and learn about TMX (https://en.wikipedia.org/wiki/Translation_Memory_eXchange)

Paul

Paul, 

enough is enough. I am giving up on that editorial thing. I just thought I would run Sublime, load the file, Sublime would do modifications, and that's it. 
Otherwise I would need to either ask for further assistance or forget about my unretrievable source (tmx) TMs. 
Now it is the highest time I were expressing my both appreciation and gratitude for your ardeous work on casting some light for me on the issue at hand. Now I truly need a break.
Best of luck Paul.