Hi Paul

I’m retagging you as it looks like Jerzy’s tag isn’t dynamic. Anyhow, I tried the suggestion you made about a year ago here (community.sdl.com/.../segmentation-rule-for-manual-line-breaks-not-working---seemingly-due-to-other-formatting-tags-studio-2019-docx), namely, the use of [\r\n]+. However, I get the exact same result as when using [\n]+.

Would you have any idea how to force a break around a soft break surrounded by tags? In a reply to Jerzy, I’m mentioning Advanced Display Filter 2.0, which uses some sort of method to filter on tags only (“Segments with tags”). Could there be a hint in the method used by ADF2?

I redid some testing with the following rules, changing only one variable at a time. Nevertheless, every time, I get the same exact number of source segments with soft breaks…

.[\n]+  |  [soft break (pasted)]  |  [Anything]   →   87 segments

[\n]+  |  [soft break (pasted)]  |  [Anything]   →   87 segments

[\r\n]+  |  [soft break (pasted)]  |  [Anything]   →   87 segments

[\r\n]+  |  [soft break (pasted)]  |  (?:.|\n)   →   87 segments

Yannic Piché

Please can you provide a source file, just a small one with only a few segments containing tags as you have shown here and that you are unable to segment as you like.  I could guess and create something similar, but sometimes the source file can be important and can also offer different ways to solve a problem depending on what the source is.  So the full picture is helpful :-)

Paul

Thank you for your reply.

I have prepared a test file (uploaded below) from the original file. I have also uploaded a screenshot of the segmentation I get.

My soft break segmentation rule settings in the only TM attached to my project are “Anything” for both “Before break” and “After break.” When I hit “Advanced View,” “[\n]+” is shown under “Before break,” and “.” is shown under “After break.”

In the Excel file type, I have enabled “embedded content processing” and added the tag definition rule “\[.+?\]” so as to make all square brackets and their contents non-translatable.

Ideally, I’d like to see no soft breaks at all (meaning I wouldn’t even see tags 0, 5, 7, 12, 15, 18, 19, 20, 22, 24, 28, and 29), with all “LINES” shown separately in their respective segments. That’s what I would have expected the behaviour of this soft break segmentation rule to be anyhow…

I’m using 2021 SR1.

Thanks again for taking the time to look at this!

TestFile.xlsx 

Yannic Piché

I can achieve this:

But I don't use segmentation rules on the TM to do this.  Instead I simply exclude the tags in the filetype like this:

Default rule plus a square bracket rule:

Then exclude the default rule:

Paul

Nifty little setting I wasn’t aware of! Thanks!

It does take care of the specific segments I provided, and my source text is now completely free of any soft breaks. However, the original file also has a bunch of sentences that have formatting tags, so they now end up broken into multiple segments (segments 12 to 16 in picture below; new test file also uploaded), whereas I would like those to stay together.

In the best possible scenario, I would want to get what’s below right away without having to resort to merging segments.

By using the “May Exclude” setting instead of “Exclude” under “Segmentation hint” and keeping my soft break segmentation rule in the TM, I get what’s below, which, in this particular case (file containing 28,000 words in 2,900 segments), appears to be what would require the least amount of manual handling (all together, I’m down to 11 segments with soft breaks instead of my initial 87 segments).

Would there be a way to get what’s in the second picture in one go? I was thinking there might be a way to force Studio to exclude only certain tags (e.g. <li>, <ul>, <p>) while always including others (e.g. <b>, <i>). Do you think there is potential there?

TestFile2.xlsx

Paul

Nifty little setting I wasn’t aware of! Thanks!

It does take care of the specific segments I provided, and my source text is now completely free of any soft breaks. However, the original file also has a bunch of sentences that have formatting tags, so they now end up broken into multiple segments (segments 12 to 16 in picture below; new test file also uploaded), whereas I would like those to stay together.

In the best possible scenario, I would want to get what’s below right away without having to resort to merging segments.

By using the “May Exclude” setting instead of “Exclude” under “Segmentation hint” and keeping my soft break segmentation rule in the TM, I get what’s below, which, in this particular case (file containing 28,000 words in 2,900 segments), appears to be what would require the least amount of manual handling (all together, I’m down to 11 segments with soft breaks instead of my initial 87 segments).

Would there be a way to get what’s in the second picture in one go? I was thinking there might be a way to force Studio to exclude only certain tags (e.g. <li>, <ul>, <p>) while always including others (e.g. <b>, <i>). Do you think there is potential there?

TestFile2.xlsx

Yannic Piché

of course. I used a catch all for the tags as I was only looking at the breaks. But you can do whatever you like with the tags and get as granular as you like.

Paul

Awesome! I’ll play around with this, see if I can make it work just as I want. Thanks for the hint with these advanced properties for the Tag Properties!! This is something I didn’t know existed, but that might just come in really handy with the types of files I often have to deal with.

Yannic Piché

Seeing as you provided the file I had a quick play... couldn't help myself ;-)

Just used these rules:

So definitely possible ;-)

Paul

Nice!

Here’s where I’m at right now. So far, I seem to get the results I expect to see in the file, but I’m sure you can notice that my knowledge of Regex is sketchy, at best.  

I’ll definitely play around with your suggestions to try to simplify and/or improve my codes!

Thanks again for sharing your knowledge!

Yannic Piché said:

my knowledge of Regex is sketchy, at best. 

As long as it works!  That's the main thing.

Paul

Suggestion(?)

It would be nice to see the type of segmentation hint shown next to the “Translate” column. Since it always has a value (Include, Include With Text, Exclude, May Exclude), I think it’d be helpful to have that value at hand, especially when setting multiple rules, instead of having to dig down to double-check your values. Would that make any sense?

Yannic Piché

Yannic Piché said:

It would be nice to see the type of segmentation hint shown next to the “Translate” column.

I agree.  Why don't you suggest it at http://ideas.sdl.com

Paul

Paul said:

Why don't you suggest it at http://ideas.sdl.com

Done.