[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: AW: RE: [xliff] RE: How to translate text within G tags?
Dear Frederik, I'm currently investigating the tagging mechanism we want to add in PASSOLO and I try to adopt the concepts of XLIFF. And in fact I currently try to figure out if it is better to store the replaced data within the tag or reference it with an id in the original segment. It is easier to understand how auto-translation can work with bpt/ept/ph but I also see that editing seems to be easier with g/bx/ex/x. And I'm not sure that using g/bx/ex/x will not work for auto-translation, or maybe yes I do! Suppose you have something like: A completely <x id=1 ctype='x-html-img> useless text. I can translate it to: Ein vollkommen <x id=1 ctype='x-html-img> unnützer <x id=1 ctype='x-html-img> Text. By cloning the image it still would be lets say valid HTML. No reason not to allow this. Now if this becomes as you described it, the source segment and is stored in the TM, then we have the situation that the tags will not match any more. Because my translation converted from the native format to XLIFF would look like this: Ein vollkommen <x id=1 ctype='x-html-img> unnützer <x id=2 ctype='x-html-img> Text. BTW: For our implementation in PASSOLO this will not be an issue because we always store the original content and add the mark-up for display and editing. The difficult thing is that we loose order information directly after editing and storing in the native file. But I hope I will be able recalulate the matching tags on the fly both for further editing and for exporting to TM. I'm thinking while I'm writing! The only issue I see in the example above is that we might not get a 100% match. The translation (our old source) has just one match and this is the correct one. Right? If we look at the original language pair I feel it is better to store: Ein vollkommen <x id=1 ctype='x-html-img> unnützer <x id=1 ctype='x-html-img> Text. (duplicated id) than: Ein vollkommen <x id=1 ctype='x-html-img> unnützer <x id=2 ctype='x-html-img> Text. (unique ids) Because if I look up a translation and get the second variant, I dont have a reference in my source skeleton to resolve the reference!!! So, if tags are not clonable we do not have problems (until we find them;-) but if we have clonable tags the replace content should be stored within the tags. But... I think SDL is not able to store tag content but just references right? OK, waiting for good ideas... Florian -----Ursprüngliche Nachricht----- Von: Corneliusson, Fredrik [mailto:Fredrik.Corneliusson@lionbridge.com] Gesendet: Donnerstag, 9. März 2006 16:23 An: florian@passolo.com; Doug Domeny; Rodolfo M. Raya Cc: bryan.s.schnabel@exgate.tek.com; xliff@lists.oasis-open.org Betreff: RE: RE: [xliff] RE: How to translate text within G tags? Hi Florian, You're right about that this is a problem. But as long as the tags in source are uniqe this is not a problem. The problem occurs if you switch the languages (TM en-de -> TM de-en) and end up with source segment with cloned tags. It's not such a big problem if you use the bpt/ept/ph approach as they provide the source tags themselves and don't need to reference the skeleton (just increment the id for added tags). This is what I like with this approach, it keeps filter/format complexity to a minimum. But with <g <bx <ex <x this is in my opinion a real problem and I think a shortcoming of the XLIFF standard. I rather feel that skeleton reference is tool/filter specific should be implemented using an attribute in the respective tools namespace. E.g. Xliff: This text is <bx id="1" myfilter:refid="1"/>special<ex id="1" myfilter:refid="2"/> and this is <bx id="2" myfilter:refid="1"/>not<ex id="2" myfilter:refid="2"/>. This would however change the definition of the "id" attribute so this is probably not an option for now. But I don't like do it the other way around either (add tool specific unique identifiers) as I think the tag reordering is useful for a lot of XLIFF tools. What's your take on this? Cheers, Fredrik -----Original Message----- From: Florian Sachse [mailto:florian@passolo.com] Sent: den 9 mars 2006 13:59 To: Corneliusson, Fredrik; 'Florian Sachse'; 'Doug Domeny'; 'Rodolfo M. Raya' Cc: bryan.s.schnabel@exgate.tek.com; xliff@lists.oasis-open.org Subject: AW: RE: [xliff] RE: How to translate text within G tags? Hi all, Dear Fredrik, yes you are right, the id is required. But I think that we cannot enforce the id to be unique. How to deal with cloneable tags? I would assume that if a source tag is used more than once in the target file, all the cloned tags should have the same id. Best regards from Bonn, Florian -----Ursprüngliche Nachricht----- Von: Corneliusson, Fredrik [mailto:Fredrik.Corneliusson@lionbridge.com] Gesendet: Mittwoch, 8. März 2006 11:16 An: Florian Sachse; Doug Domeny; Rodolfo M. Raya Cc: bryan.s.schnabel@exgate.tek.com; xliff@lists.oasis-open.org Betreff: RE: RE: [xliff] RE: How to translate text within G tags? Hi all, >> Regarding the ids I think, they are important only for the extraction/merger tool to retrieve the original content replaced by the tag. If the original content can completely be retrieved from the tag, the there might not be a need the the id. So it could be optional. I think the id attribute for inline tags is extreamly important, and there value need to be uniqe in the segment (apart from paied tags that can share the same id). When using a Translation Memory you need to uniquely identify tags to be able to reorder the tags and have the application move the correct source tag into target. First occurrence: S: This text is <b>special</b> and this is <b>not</b>. T: This is <b>not</b> special and this <b>is</b>. As you can see for the TM to rearrange the tags automatically it needs to know the unique id for tags. XLIFF: This text is <bx id="1"/>special<ex id="1"/> and this is <bx id="2"/>not<ex id="2"/>. Second occurrence: S: This text is <b>special</b> and this is <del>not</del>. T: This is <del>not</del> special and this <b>is</b>. If the TM engine knows the order of the tags it's easy to replace the tags to reflect the new source. Regards, Fredrik -----Original Message----- From: Florian Sachse [mailto:florian@passolo.com] Sent: den 7 mars 2006 23:04 To: Doug Domeny; 'Rodolfo M. Raya' Cc: Corneliusson, Fredrik; bryan.s.schnabel@exgate.tek.com; xliff@lists.oasis-open.org Subject: AW: RE: [xliff] RE: How to translate text within G tags? Hi all, Better late than never. Inline tags are not so easy. I'm currently on a trip to Dublin but want to share some thoughts with you: The XLIFF spec is not describing the usgae of inline tags very well. You find for example in all elements bold as an example of the content. You see I have my spec with me all the time;-) From my own work with mark-up I came up with the following tag types: 1. A tag marking up almost not translateble text like placeholders in software (see Java profile or html element img). Sub elemts can be used inside this tag to mark up localizable text (alt attribute in hmtl img, choice in Java placehoder. I see ph to be this tag: "localizable .. <ph>not localizable</ph>...localizable 2. A tag marking up something short which must not be localized for sure. I see x to be this tag. It can be used for example to represent a html-break: Localizable text...<x ctyle="lb">...localizable text 3. A tag that can be used to mark-up localizable text (like bold) in a way that tags do not overlap. This is where I see the g tag. 4. A tag that can be used to mark-up localizable text with tags that can overlap, like in html. This is something for bx and ex. These are thr four different cases I think which must be covered. Regarding the ids I think, they are important only for the extraction/merger tool to retrieve the original content replaced by the tag. If the original content can completely be retrieved from the tag, the there might not be a need the the id. So it could be optional. For bx ex xid is mandatory because it helps to identify the matching start and end tags. The number of tags also must not be equal in source and target, like bold sections or breaks in html. Others might be. This can be controled by the attribute clone. As recommendations I would say Use the simplest meachnism to markup. X instead of ph. G instead of bx ex. If tags are broken during segmentaion, duplicate the missing tags at the beginning or end of the segments. I think tmx is proposing something more complicated. This could by the way conflict with the clone attribute. Any ideas here? Thnaks and best regards from my Air Lingus flight, Florian ----- Ursprüngliche Nachricht ----- Von: "Doug Domeny" <ddomeny@ektron.com> An: "'Rodolfo M. Raya'" <rodolfo@heartsome.net> Cc: "'Corneliusson, Fredrik'" <Fredrik.Corneliusson@lionbridge.com>; bryan.s.schnabel@exgate.tek.com; xliff@lists.oasis-open.org Gesendet: 07.03.06 17:59 Betreff: RE: [xliff] RE: How to translate text within G tags? Rodolfo, Thanks again for your input. My comments are below. Regards, Doug Domeny Software Analyst Ektron, Inc. +1 603 594-0249 x212 http://www.ektron.com _____ From: Rodolfo M. Raya [mailto:rodolfo@heartsome.net] Sent: Tuesday, March 07, 2006 10:52 AM To: Doug Domeny Cc: 'Corneliusson, Fredrik'; bryan.s.schnabel@exgate.tek.com; xliff@lists.oasis-open.org Subject: RE: [xliff] RE: How to translate text within G tags? On Tue, 2006-03-07 at 09:13 -0500, Doug Domeny wrote: Hi, ORIGINAL SOURCE Italic texts starts <i><b>in the middle of first sentence</b>. Italics ends after the second sentence.</i> XLIFF SOURCE <source>Italic texts starts <bpt id='i1' ctype='x-html-i'/><btp id='b1' ctype='x-html-b'/>in the middle of first sentence<ept id='b1' ctype='x-html-b'/>. Italics ends after the second sentence.</ept id='i1' ctype='x-html-i'></source> XLIFF TARGET <target>Italic texts starts <bpt id='i1' ctype='x-html-i'/><btp id='b1' ctype='x-html-b'/>in the middle of first sentence<ept id='i1' ctype='x-html-i'/><ept id='b1' ctype='x-html-b'/>. <bpt id='i1' ctype='x-html-i'/><btp id='b1' ctype='x-html-b'/>Italics ends after the second sentence.<ept id='i1' ctype='x-html-i'/></target> This is wrong. Target should not have more tags than source text. [doug] It is wrong, but it is well-formed XML. I'm not sure if an XML schema could detect it. I'll look into it later. My concern is that a tool would produce this incorrect tagging. Even if every btp had a matching ept following it, overlapping tags (e.g., <b><i></b></i>) are a problem in XHTML/XML, but not RTF. So in effect, XLIFF should allow them, but they would result in bad XML. MERGED TRANSLATION Italic texts starts <i><b>in the middle of first sentence</i></b>. <i><b>Italics ends after the second sentence.</i> Notice that <i> and <b> overlap and that a closing <b> is missing even though the contents of the <target> tag are well-formed. It does not make sense to add an opening <bpt> (blue one) in the second sentence. Notice that it does not have a matching <ept> in your sample. [doug] Yes, it does not make sense, but people make mistakes. I've received corrupted HTML back from translators and I'm concerned that translators would be able to move and/or copy and paste <bpt> and <ept> tags. I'm using 'id' in <g> to reference the skeleton. I'm concerned that segmentation will cause problems with referencing the skeleton. To illustrate, please consider the example from above. ORIGINAL SOURCE Italic texts starts <i><b>in the middle of first sentence</b>. Italics ends after the second sentence.</i> XLIFF SOURCE <source>Italic texts starts <g id='1' ctype='x-html-i'><g id='2' ctype='x-html-b'>in the middle of first sentence</g>. Italics ends after the second sentence.</g></source> where '1' reference <i> and '2' references <b>. XLIFF TARGET SEGMENTED <target>Italic texts starts <g id='1' ctype='x-html-i'><g id='2' ctype='x-html-b'>in the middle of first sentence</g></g>. <g id='1' ctype='x-html-i'><g id='2' ctype='x-html-b'>Italics ends after the second sentence.</g></g> </target> I don't understand why you close the red <g> in the first sentence and reopen it in the second one. This methodology may crash with target languages like Chinese or Arabic. [doug] I agree this is a contrived example. I attempted to show that segmentation could require duplicating inline tags. Perhaps this example is better: <source>Housing prices are <g id='1' ctype='bold'>rising. White</g> houses are popular.</source> <target>Los precios de la vivienda <g id='1' ctype='bold'>suben</g>. Las casas <g id='1' ctype='bold'>blancas</g> son populares.</target> Note that the tags are duplicated so there are two <g> tags with id='1' and two with id='2'. There are two <g> tags that map to one <i> in the skeleton and two to one <b>. This scenario precludes merging the target text with the skeleton for inline tags that have been duplicated as a result of segmentation or reordering. Perhaps the target text should not be merged with the skeleton, but simply reconstructed. This would be a blending of the minimal (with skeleton) and maximal (no skeleton for inline tags) approach. Reconstructing the target is not a good idea. This may work for some languages, but not all. I'm quite sure that you will have troubles handling Hebrew, Arabic and Chinese. [doug] 'Reconstructing' may be the wrong word. Currently, my approach is to match, one-to-one, the inline tags in the <target> with the original HTML inline tags in the skeleton. Your example of bidirectional languages is well taken. But I'm not sure how the translator would indicate directionality. Wouldn't <span dir='rtl'> tags need to be added? I'm left with a bit of a dilemma. If the translator can add or duplicate inline tags in the target, then there isn't a one-to-one correspondence between the target and the skeleton. I'm not sure how to merge elements in the target with those in the skeleton. On the other hand, if the output is simply created from the target without a skeleton, then some information may be lost. Here's another example, I did <font color='red'>not</font> enter <source>I did <g id='1' ctype='x-html-font'>not</g>enter.</source> <target>Je <g id='1' ctype='x-html-font'>ne</g> suis <g id='1' ctype='x-html-font'>pas</g> entré.</target> (My apologies to French-speakers if my literal translation of 'not' to 'ne pas' is wrong, but hopefully it shows the possibility of duplicating tags) Now there are two text nodes 'ne' and 'pas' that can't be merged where the 'not' is in the original. I'm seeing that my current approach won't work. The following is obviously wrong. It processes the skeleton and draws translated text from the <target> Je <font color='red'>nepas</font> suis entré. Merging the other direction may work. I'll need to try it. Perhaps someone has already solved this problem. The following processes the <target> and copies tag attributes, etc, from the skeleton. It works in this case, but there may be cases I haven't considered. Je <font color='red'>ne</font> suis <font color='red'>pas</font> entré. If someone has already figured this out, please let me know and we should also add it to the HTML profile. BTW, although our focus has been on XHTML and XML, the Ektron CMS collects related text together into one XLIFF file. For example, there may be several blocks of XHTML content along with user-defined meta-data and Ektron CMS meta-data needed to import the translated content back into the system. The conversion to TMX seems worth considering too. Conversion to TMX is crucial. I have routines that map XLIFF tags to TMX and from TMX to XLIFF. The content of the tag is vital and with <g> elements used to hold translatable text conversion becomes too complex, if not impossible. A <g> tag that you identify as holding italics in XLIFF does not contain the inline codes that should be placed in the TMX counterpart. In a TMX file you enclose formatting code, like "\i" or "<i>" within an inline element. That is the information that is exchanged. The use of <g> as suggested in the HTML profile does not include the formatting in the XLIFF file and this makes exporting translated and approved segments from XLIFF to TMX too complicated, specially if the translator doing the conversion does not have the skeleton at hand. I hope this message reaches the mailing list. The replies I sent yesterday still don't appear in OASIS web site and I did not get a copy back from the server. Best regards, Rodolfo Fortunately, none of these issues seem insurmountable. It's mostly a matter of clearing up ambiguities as we resolve interoperability issues and establish best practices. Regards, Doug Domeny Software Analyst Ektron, Inc. +1 603 594-0249 x212 http://www.ektron.com _____ From: Corneliusson, Fredrik [mailto:Fredrik.Corneliusson@lionbridge.com] Sent: Tuesday, March 07, 2006 5:15 AM To: bryan.s.schnabel@exgate.tek.com; ddomeny@ektron.com; rodolfo@heartsome.net Cc: xliff@lists.oasis-open.org Subject: RE: [xliff] RE: How to translate text within G tags? Hello, I just joined and this is my first post! My XLIFF experience is mostly as a XLIFF Editor/filter programmer (Transolution). I must say that from my point of view I much prefer the <bpt/ept way of wrapping inline tags, and if the editor has tag checking it's easy to check that they are valid. I had the same problem with deciphering the use of <g tag from the spec as Rodolfo, and until I read the "XLIFF 1.2 Representation Guide for HTML" I was hoping I never had to deal with them as containing translatable content. XLIFF is quite a lot to digest and the <g tag really doubles the effort as it breaks the simple logic that can be used on a flat structure for translatable content. Also at some time you will need to convert XLIFF to TMX and then you need to convert it to <bpt/ept anyway. Using ph/bpt/ept gives you a very generic and straight forward approach and you preserve the original source format information exactly as it is and you can treat all formats the same. That said I can see why people like the <g approach. It's easier to wrap in existing translation tools and process with XSLT, it also looks nicer in a text editor and I suppose lessens the need for skeleton files. I have implementation question regarding the <g tag, in the XLIFF documentation the specification of the g-tags "id" attribute is different to that of the ph/bpt/ept: ph-tag: The required id attribute is used to identify the <ph> inline code g-tag: The required id attribute is used to reference the replaced code in the skeleton file. Does this mean that there can be <g and <ph tags with the same id in a segment? And what if there is no skeleton file? This brings me to a general complaint about the XLIFF spec, it is very vague and leaves a lot of room for personal taste and/or misunderstandings. This makes if hard to create a generic editor that works with XLIFF's in the wild. For example TU's have required ID attribute but it can be anything and does not even have to be unique, so why is it required in the first place? Cheers, Fredrik Corneliusson _____ From:bryan.s.schnabel@exgate.tek.com [mailto:bryan.s.schnabel@exgate.tek.com] Sent: den 7 mars 2006 01:26 To: ddomeny@ektron.com; rodolfo@heartsome.net Cc: xliff@lists.oasis-open.org Subject: RE: [xliff] RE: How to translate text within G tags? Hi Doug, I thought about this when I wrote that portion of the HTML profile. From a philosophical view, I strongly think I bpt/ept should only be used in XLIFF files that are derived from non-markup formats (RTF, for example). I really don't like the idea of using bpt/ept on XLIFF files derived from HTML, XHTML, or XML files. I see "begin paired tag" and "end paired tag" as an artificial device. It could easily lead to malformed XML on the conversion from XLIFF back to HTML. Assuming the source file is well formed, it would be a shame to have to delimit inline elements in an artificial way. If <g tags are defined in the spec in such a way that they are thought to be for non-translatable text, I would vote to either update the specification, or come up with a new element for identifying translatable inline elements in <target elements. Thanks to Doug and Rodolfo for brining this issue to light, Bryan -----Original Message----- From: Doug Domeny [mailto:ddomeny@ektron.com] Sent: Monday, March 06, 2006 3:29 PM To: 'Rodolfo M. Raya' Cc: xliff@lists.oasis-open.org Subject: [xliff] RE: How to translate text within G tags? Rodolfo, others, Yes, I can see where <g> could be interpreted as containing only non-translatable codes. I based my use of them on the HTML profile document (see below). I've copied the rest of the XLIFF cmte for their input. Should the HTML profile be revised to use bpt/ept instead of <g> for inline elements? 3.1.4. Inline Elements In most cases, inline elements are very well suited to be mapped to <g>. The value of their ctype attribute should be a concatenation of 'x-html-' and the name of the element (in lowercase). For example: <b> would be mapped to <g ctype='bold'>. Example: <p>In Portland, Oregon one may <i>ski</i> on the mountain, <b>wind surf</b> in the gorge, and <i>surf</i> in the ocean, all on the same day.</p> should be mapped to: <trans-unit id='1' restype='x-html-p'> <source xml:lang='en'>In Portland, Oregon one may <g id='i1' ctype='x-html-i'>ski</g> on the mountain, <g id='i1' ctype='bold'>wind surf</g> in the gorge, and <g id='i1' ctype='x-html-i'>surf</g> in the ocean, all on the same day.</source> </trans-unit> Regards, Doug Domeny Software Analyst Ektron, Inc. +1 603 594-0249 x212 http://www.ektron.com _____ From: Rodolfo M. Raya [mailto:rodolfo@heartsome.net] Sent: Monday, March 06, 2006 7:01 PM To: Doug Domeny Subject: Re: How to translate text within G tags? Hi Doug, As far as I know, <g> elements contain inline codes, not translatable text. >From the specs: http://www.oasis-open.org/committees/xliff/documents/xliff-specification.htm #g <http://www.oasis-open.org/committees/xliff/documents/xliff-specification.ht m> <g> Generic group placeholder - The <g> element is used to replace any inline code of the original document that has a beginning and an end, does not overlap other paired inline codes, and can be moved within its parent structural element. To me, this means that <g> is used to wrap moveable inline code and for "inline code" I understand tags inherent to the formatting of the document, i.e. "\i" and "\i0" to mark start and end of italics in RTF or "<i>" and "</i>" for the same purpose in HTML. IMO, only the formatting portion can be enclosed in <g>. Notice that <g> cannot contain any <sub> element with translatable text inside. In your example I would enclose the text "no need to download again in a <bpt>/<ept> pair. According to the introduction of the specs, XLIFF "borrows" elements from TMX and you can read a clear example of handling italics in TMX at http://www.lisa.org/standards/tmx/tmx.html#ContentMarkup_Rules If you consider <g> as a black box with translatable text that can be moved, the translatable text that it contains may end at any location in the translated segment. The result can be horrible. Think on RTF documents that when converted to XLIFF contains tags in the middle of a word in the source text. Most of the times those tags are originated in change tracking and signal the correction of a spelling error. Translators usually place those tags at the end of the segment or after the corresponding word in their translations but never in the middle of the equivalent word in the target language. This is the typical use case for <g> tags. Best regards, Rodolfo On Mon, 2006-03-06 at 17:20 -0500, Doug Domeny wrote: Rodolfo, While testing with the XLIFF Editor, I couldn't find a way to translate text between <g> tags. For example, How would a translator translate the phrase "not need to download again"? The XLIFF is: <target state="needs-translation">Before you can use eWebEditPro, it must be downloaded into your browser. When you click the <g id="2" ctype="bold">Install Now</g>button at the bottom of this page, eWebEditPro will be automatically downloaded and installed. This process may take several minutes depending on the speed of your network connection. Once downloaded, eWebEditPro will <g id="4" ctype="italic">not need to download again</g> unless upgrading to a newer version.</target> Regards, Doug Domeny Software Analyst Ektron, Inc. +1 603 594-0249 x212 http://www.ektron.com -- The information in this e-mail is intended strictly for the addressee, without prejudices, as a confidential document. Should it reach you, not being the addressee, it is not to be made accessible to any other unauthorised person or copied, distributed or disclosed to any other third party as this would constitute an unlawful act under certain circumstances, unless prior approval is given for its transmission. The content of this e-mail is solely that of the sender and not necessarily that of Heartsome. -- The information in this e-mail is intended strictly for the addressee, without prejudices, as a confidential document. Should it reach you, not being the addressee, it is not to be made accessible to any other unauthorised person or copied, distributed or disclosed to any other third party as this would constitute an unlawful act under certain circumstances, unless prior approval is given for its transmission. The content of this e-mail is solely that of the sender and not necessarily that of Heartsome.
<<attachment: winmail.dat>>
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]