utf8 in fromxsf - inconsistent output

Question

Hello. I have an issue with the -utf8 switch not working consistently for the fromxsf command. 
 My input data contains a section sign: &sect; . 
 I am running this command on the DIV composed from that data: fromxsf -utf8 -cd $DIVpath $OUTfile 
 For one of my DIVs, the section sign outputs as " &sect; ". For another DIV, it outputs as unicode - i.e. " &#167; "; 
 If I switch the data around between these DIVs, one still consistently outputs " &sect; " and the other consistently outputs " &#167; " . 
 I can only conclude the issue is not with the data but instead there is something in the "bad" DIV that does not recognize the -utf8 switch. 
 Any ideas why this is happening? 
 All help is appreciated. 
 Thank you.

Jonathan Dagresta · Answer

My guess is that you're talking about XML data and in one of your DIVs the data came in (via toxsf) with the character specified as a numeric character reference, namely "&#167;". 
 An excerpt from the on-line help file for fromxsf says: Characters which are input (via toxsf) as entities or as numeric character references are always output unchanged. 
 Specifying the -utf8 option does not change that behavior. XPP is trying its best to maintain the data as originally input when round-tripping the data (at least in this aspect). 
 You should be able to tell with the xyview by finding the section sign character and opening the Line Editor and you'll see the character show as the "&#167;" sequence rather than just as the section sign character. If you cursor past the sequence in the Line Editor, you'll see the cursor jump past the whole sequence indicating that it's representing a single character. All of that indicates that the data came in that way. 
 I hope that helps explain what you are seeing.

XPP > XPP

utf8 in fromxsf - inconsistent output