Fw: [xliff] Segmentation Modifications

[Helena S Chapman <hchapman@us.ibm.com>]

FYI Response from w3c.

----- Forwarded by Helena
S Chapman/San Jose/IBM on 12/16/2013 09:44 AM -----

From:      
 "Aharon (Vladimir)
Lanin" <aharon@google.com>
To:      
 Steven R Loomis/Cupertino/IBM@IBMUS
Cc:      
 Richard Ishida <ishida@w3.org>,
Helena S Chapman/San Jose/IBM@IBMUS, Michael Ow/Southbury/IBM@IBMUS
Date:      
 12/15/2013 05:50 AM
Subject:    
   Re: Fw: [xliff]
Segmentation Modifications

---

Caveat: I know very little about XLIFF outside of what
I found in the Wikipedia article and some parts of the spec before writing
this email.

I think that the place to start is to take a step backwards.
The basic reason that we have directionality in text is that different
scripts have different directionalities. To put it another way, 99% of
the time, the directionality of a piece of text stems from the script in
which it is written. Thus, for example, Hebrew text is RTL and Latin text
is LTR. Now, when I embed a Latin phrase (e.g. a movie name or a street
address) in some Hebrew text, or vice versa, it is important to indicate
the directionality of the embedded text, since otherwise the embedded text
has a good chance of not being displayed as intended (e.g. "19 Main
Street, Oakland" when displayed RTL comes out looking like "Main
Street, Oakland 19" - which makes as little sense in a Hebrew document
as it does in an English one). And thus, we have the directional formatting
characters of Unicode and the dir attribute in HTML and XML. My point,
however, is that in 99% of the cases, the reason a direction
switch is needed is that a script switch has taken place. The way
to indicate a script (in XLIFF as in HTML, XML, etc.), whether implicitly
or explicitly, is with the lang attribute. It is true that Unicode
does not have formatting characters to indicate language or script, and
the lang attribute of HTML/XML is used quite rarely, but that is only because
in most cases one does not need to know the language of a piece of text
in order to display it as intended. Without knowing the directionality,
however, one will often display the text garbled. Thus, directionality
gets indicated a lot more often than the language - but the underlying
cause of directionality changes is still a change in script, even though
most of the time it does not get indicated explicitly.

Now, in XLIFF, the only elements allowed to
have xml:lang are <source> and <target>. Furthermore:

"When a <source> element is a child of <segment>
or <ignorable> and the OPTIONAL xml:lang attribute is present, its
value MUST be equal to the value of the srcLang attribute of the enclosing
<xliff> element."
"When a <target> element is a child of <segment>
or <ignorable> and the OPTIONAL xml:lang attribute is present, its
value MUST be equal to the value of the trgLang attribute of the enclosing
<xliff> element."

Thus, effectively, the entire XLIFF document has just
one source language and just one target language.

In contrast, XLIFF restricts the value of the dir attribute
on neither <source> nor <target>. Furthermore, the dir attribute
is also allowed on <data>, <pc> and <sc>.

I do not understand this.

If the <source> of segment 1 and the source of segment
2 are in the same language and script, why would one need to be LTR while
the other needs to be RTL?

The only ghost of a reason I can think of is that the
LTR segment is actually a mathematical formula or something like it, since
in several RTL language mathematics are nevertheless written LTR, but I
have trouble believing that this is really the reason for this anomaly
in XLIFF (or that there isn't a better way of indicating that a segment
is actually a mathematical _expression_ than by using the dir attribute).

In any case, assuming that there is a good reason, if
the dir attribute is available on <pc>, and <pc dir="ltr|rtl">
basically has the same meaning as <span dir="ltr|rtl">
in HTML, then why not indicate the directionality of segments that got
combined by adding a <pc> around the text from each one?

What is the purpose of the dir attribute on <data>?

Aharon


On Fri, Dec 13, 2013 at 9:18 PM, Steven R Loomis <srloomis@us.ibm.com>
wrote:
Richard, Aharon,
 FYI The  XLIFF  translation standard is trying to come
up with recommendations around their use of directional control chars vs.
markup.  Would either of you be able to provide some input? I can
see if they can put together some of their questions.

Thanks,
Steven

----- Forwarded by Steven R Loomis/Cupertino/IBM on 13/12/2013 11:16 -----

From: "Dr. David Filip"
<David.Filip@ul.ie>
To: Yves Savourel <ysavourel@enlaso.com>
Cc: Steven R Loomis/Cupertino/IBM@IBMUS,
"xliff@lists.oasis-open.org"
<xliff@lists.oasis-open.org>
Date: 13/12/2013 06:36
Subject: Re: [xliff] Segmentation
Modifications

---

Stephen, Yves, Fredrik, all,

I was looking up the bidi algorithm UAX#9, and I am not sure if we should
be using the explicit directionality control characters. The UAX#9 itself
quotes UTR#20, http://www.w3.org/TR/unicode-xml/ which
discourages the use of control characters in markup environment.

I know that XLIFF 1.2 did not have anything else, but why not have a full
markup solution this time round..

I wonder if we should rather use directionality annotations based on markers,
or dedicated directionality elements.

Another related issue is that both Unicode 6.3 and HTML 5 now allow for
heuristic determination of the directionality by the first strong character,
and there might be cases where this cannot be resolved into an explicit
directionality becuase of varaibales..

So whether we use control characters or if we go for marker based directionality
markup or even for dedicated directionality elements similar to HTML bdi,
we should have a value equivalent to FSI and bdi="auto"

Rgds
dF

Dr. David Filip
=======================
LRC | CNGL | LT-Web | CSIS
University of Limerick, Ireland
telephone: +353-6120-2781
cellphone: +353-86-0222-158 
facsimile: +353-6120-2734
http://www.cngl.ie/profile/?i=452
mailto: david.filip@ul.ie


On Thu, Dec 12, 2013 at 6:41 PM, Yves Savourel <ysavourel@enlaso.com>
wrote: 
Thanks Steven,

Exactly the type of feedback I was looking for.

 

So we should do RLI+PDI and LRI+PDI instead of RLE+PDF and LRE+PDF  I suppose?

 

-ys

 

From: Steven R Loomis [mailto:srloomis@us.ibm.com]
Sent: Thursday, December 12, 2013 10:55 AM

To: Yves Savourel
Cc: xliff@lists.oasis-open.org
Subject: RE: [xliff] Segmentation Modifications

 

Jumping in here..
 Please note that Unicode 6.3 adds directional isolate characters, which could be useful for joining segments.

See:  http://www.unicode.org/reports/tr9/#Directional_Formatting_Characters 

Directional isolate characters were introduced in Unicode 6.3 after it became apparent that directional embeddings usually have too strong an effect on their surroundings and are thus unnecessarily difficult to use. The new characters were introduced instead of changing the behavior of the existing ones because doing so might have had an undesirable effect on those existing documents that do rely on the old behavior. Nevertheless, the use of the directional isolates instead of embeddings is encouraged in new documents – once target platforms are known to support them.

-s


Yves Savourel ---12/12/2013 05:50:43---For reference, the bidi text I’m talking about is this one: [[

From: Yves Savourel <ysavourel@enlaso.com>
To: <xliff@lists.oasis-open.org>
Date: 12/12/2013 05:50
Subject: RE: [xliff] Segmentation Modifications
Sent by: <xliff@lists.oasis-open.org>

---

For reference, the bidi text I’m talking about is this one:

[[
If the dir attributes of the <source> or <target> elements differ: The content of the <source> or <target> elements set to a
different directionality than the directionality for the <source> or <target> elements of the joined segment MUST be enclosed
between Unicode bi-directional control characters reflecting their original directionality (U+202A and U+202C for left-to-right
spans, and U+202B and U+202C for right-to-left spans).
]]

From the attached file in this post:
https://lists.oasis-open.org/archives/xliff/201311/msg00176.html

The question is basically: are those Unicode control characters the one to use for this mapping?

I based the text on this article:
http://www.w3.org/International/questions/qa-bidi-controls


Thanks,
-yves


From: Yves Savourel [mailto:ysavourel@enlaso.com]
Sent: Thursday, December 12, 2013 6:04 AM
To: 'xliff@lists.oasis-open.org'
Subject: RE: [xliff] Segmentation Modifications

Hi David,

I can do the change, that will free you time for other ones.

Did you double check the bidi mapping?
I’m not expert on bidi, so it’d be good to have more than my input on that part.

Cheers,
-yves

From: Dr. David Filip [mailto:David.Filip@ul.ie]
Sent: Thursday, December 12, 2013 5:48 AM
To: Yves Savourel
Cc: xliff@lists.oasis-open.org
Subject: Re: [xliff] Segmentation Modifications

Yves, all I did not hear any dissent on that

As far as i checked this, your proposal is equivalent to what was there for csprd02 with two small exceptions that add to clarity:

1) You use an explicit bidi provision, so that people do not need to research the Unicode BiDi algorithm for merging segments with
different dir

2) You also proposed to have an option to downgrade state on split segments, which makes sense to me

Otherwise it is is just reorganizing the PRs by the perfomred type of modification, which seems fine and I do not have a preference
regarding the presentation of the provisions.


@Yves, Do you want to implement this proposal in the spec or should I?
Please let me know

Thanks
dF


Dr. David Filip
=======================
LRC | CNGL | LT-Web | CSIS
University of Limerick, Ireland
telephone: +353-6120-2781
cellphone: +353-86-0222-158 
facsimile: +353-6120-2734
http://www.cngl.ie/profile/?i=452
mailto: david.filip@ul.ie

On Sat, Nov 30, 2013 at 1:56 PM, Yves Savourel <ysavourel@enlaso.com> wrote:
Hi all,
 
As mentioned here: https://lists.oasis-open.org/archives/xliff/201311/msg00138.html, I've been trying to implement segmentation
modification for XLIFF 2.0 for a while now and I have a few comments.
 
For reference, the cs02 section for this is here:
http://docs.oasis-open.org/xliff/xliff-core/v2.0/csprd02/xliff-core-v2.0-csprd02.html#d0e9317
 
 
--- The section (starting with its new title) keeps talking about "segmentation modification" and "resegmentation". Could we just
talk about segmentation modification everywhere? The two things are the same thing.
 
 
--- That section has many constraints and processing requirements.
It was quite difficult to follow when I tried to implement it.
 
For example: (take a deep breath) "Modifiers MUST copy all attributes including values, except for the id and order attributes, from
their original instances on or within the original <segment> element onto both instances on and within the resulting two <segment>
or <ignorable> elements, except for attributes that do not have valid instances on the eventually resulting <ignorable> element."
 
To make a long story short and get to the point, I think that section should be re-worded to be simpler, organized by action (split
or join), and completed with a few things (some subState PRs, explicit directionality conversion, etc.)
 
The proposed modified text is in the attached document.
 
I believe it covers what is needed, but it's a complex set of PRs and it should be carefully checked by all. For example I'd like a
confirmation on the Unicode control characters used for the directionality conversion.
 
Thanks,
-yves
 
 


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 



---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php