- From: Asgeir Frimannsson <asgeirf@redhat.com>
- To: Paul Gampe <pgampe@redhat.com>,Jim Hogan <j.hogan@qut.edu.au>
- Date: Wed, 11 May 2005 16:34:58 +1000
Hi Paul, Here's a fresh version of the PO Repr. Guide with OASIS formatting. Basically a re-write of what's on the wiki. cheers, asgeirTitle: XLIFF 1.1 Representation Guide for Gettext PO
Working Draft 25 April 2005
Copyright © 2005 The Organization for the Advancement of Structured Information Standards [OASIS] Table of Contents
AppendixesAs different tools may provide different filters to extract the content of Gettext Portable Object (PO) documents it is important for interoperability that they represent the extracted data in identical manner in the XLIFF document. The intent of this document is to provide a set of guidelines to represent PO data in XLIFF. It offers a collection of recommended mappings of all features of PO the PO format developers of XLIFF filters can implement, and users of XLIFF utilities can rely on to insure a better interoperability between tools. Because the Gettext PO format is not a defined standard - nor is the format well documented, we will in this section present an overview of the features and design of the PO file format. There are two types of PO files: PO Template files (POTs) and Language specific PO files (POs). POTs contains a skeleton header, followed by the extracted translation units. POTs are generated by the xgettext extraction tool and are not meant to be edited by humans. POTs are converted into Language Specific POs by the msginit tool, and these files are then edited by translators. When source code is updated, a new POT is generated for the project, and the changes from previous versions are incorporated into the existing translations by using the msgmerge tool. This tool inserts new translation units into the existing PO files, marks translation units no longer in use as obsolete, and updates any references and extracted comments. Translated PO files are converted to binary resource files, known as MO (Machine Object) files, by the msgfmt tool. The Gettext library use MO files at runtime; hence PO files are only used in the development and localisation process. A PO file starts with a header, followed by a number of translation units. # SOME DESCRIPTIVE TITLE. # Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER # This file is distributed under the same license as the PACKAGE package. # FIRST AUTHOR <EMAIL@ADDRESS>, YEAR. # #, fuzzy msgid "" msgstr "" "Project-Id-Version: PACKAGE-NAME VERSION\n" "Report-Msgid-Bugs-To: BUG-EMAIL-ADDR <EMAIL@ADDRESS>\n" "POT-Creation-Date: YEAR-MO-DA HO:MI+ZONE\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" "Language-Team: LANGUAGE <EMAIL@ADDRESS>\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=CHARSET\n" "Content-Transfer-Encoding: 8bit\n" "Plural-Forms: nplurals=2; plural=n!=1;\n" "X-User-Defined-Var: VALUE\n" # Translator Comment #. Extracted Comment #: myfile.c:12 #, flag msgid "Original String 1" msgstr "Translated String 1" # Translator Comment #. Extracted Comment #: myfile.c:23 #, flag msgid "Original String 2" msgstr "Translated String 2"
# French Translation for MyApplication. # Copyright (C) 2005 John Developer # This file is distributed under the same license as the MyApp package. # John Developer <john@example.com>, 2005. # Joe Translator <joe@example.com>, 2005. # msgid "" msgstr "" "Project-Id-Version: MyApp 1.0\n" "Report-Msgid-Bugs-To: MyApp List <myapp-list@example.com>\n" "POT-Creation-Date: 2005-04-27 13:15+0900\n" "PO-Revision-Date: 2005-04-27 13:45+0900\n" "Last-Translator: Joe Translator <joe@example.com>\n" "Language-Team: French Team <fr-list@example.com>\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" "Plural-Forms: nplurals=2; plural=(n!=1);\n" "X-Generator: KBabel 1.9\n"
The PO header follows a simliar structure to PO translation units, but is
destinguished by its empty source element (
The initial comment lines (comments are lines starting with
The header skeleton in a POT file is initially marked with the
Table 1. Predefined PO Header variables
In addition to these predefined variables, the PO header can contain custom user-defined variables of the same format.
# Translator Comment #. Extracted Comment #: myfile.c:12 myfile.c:32 #, flag msgid "Original String" msgstr "Translated String"
PO translation units use the source string (
The
The actual content of msgid "" "My name is " "" "%s. \n" "What is" " " "your name?" is exactly the same as: msgid "My name is %s. \n What is your name?"
# This is a comment line # This is another comment line
Translator comments are lines starting with
#. This is an extracted comment #. This is another extracted comment
Extracted comments are lines starting with /* This comment will be extracted */
gettext("Hello World");This would become: #. This comment will be extracted msgid "Hello World" msgstr ""
When updating a PO file from a new POT file, existing extracted comments in the language specific PO file are discarded, and the extracted comments present in the POT file are inserted in the existing PO file.
#: myfile.c:1 myfile.c:23 otherfile.c:1 #: otherfile.c:34
References are identified by lines starting with
As each Similar to extracted comments, when updating a PO file from a new POT file, existing references in the language specific PO file are discarded, and the references present in the POT file are inserted in the existing PO file.
Flags are identified by lines starting with Flags are used both as processing instructions by the Gettext tools, and by translators to indicate that a translation unit is unfinished or "fuzzy". Table 2. Flag values and descriptions
Flags (except /* xgettext:no-c-format */
printf(_("Hello World"));
Since the Gettext call here is inside a #, no-c-format msgid "Hello World" msgstr ""
Gettext, in addition to supporting normal translation units with a single msgid "You have %d file" msgid_plural "You have %d files" msgstr[0] "Du har %d fil" msgstr[1] "Du har %d filer"
The target language may have one or more forms (Japanese has one form,
while Polish has 3 forms), and the logic for selecting which form to use
for a parameter is defined in a PO header field, where "Plural-Forms: nplurals=2; plural=(n != 1);\n"
This is a typical example for a Germanic language, which has a special case
when "Plural-Forms: nplurals=3; " "plural=n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2;"
C-expressions are defined as
Obsolete entries are translation units that are no longer present in the source-files,
and are therefore commented out when a PO file is updated.
These entries are re-used by Gettext only if the translation-unit re-appears in the project,
and are also used for fuzzy matching by the 'msgmerge' tool.
Obsolete entries are marked with # This is a translator comment #~ msgid "" #~ "Please enter the following details:\n" #~ " - First Name\n" #~ " - Last Name\n" #~ msgstr "" #~ "Venligst fyll inn følgende data:\n" #~ " - Fornavn\n" #~ " - Etternavn\n"
One single PO file normally represents one MO file, known as a Gettext domain,
but the PO format also allows for representing multiple domains in a single PO file. This
is done by adding the domain "domain_1" msgid "hello world" msgstr "hei verden" domain "domain_2" msgid "hello world" msgstr "hei verden" The above example would produce two MO files, domain_1.mo and domain_2.mo.
If no domain is specified, translation units belong to the default domain messages.
A PO header is bound to a domain, so each domain has its own header. Having mulitple domains in a single PO file is very rare; in fact, the authors have never seen this in use. This section discusses the general considerations to take in account when extracting data from PO files. Because of good open source tool support, the PO file format has been used as a common file format for the extraction of localisable data from a number of different source formats, including XML-based document-formats such as Docbook. This guide mainly covers representation of PO files generated from the GNU Gettext toolkit - targetting only localisation of software messages. It is fully possible to apply this guide to PO files extracted from XML formats. However, it is highly recommended to use native XLIFF filters wherever possible, and not use PO as a middle-format in these processes.
The PO file format does not provide a way of identifying the source and target
language within a file. By GNU standards, GNU software is written in American
English (
POSIX locale names typically use the form
Locale names (through use of the The PO file format is different from most other software localisation resource formats in that it does not use ID based translation units. Gettext use the source string as the primary id, meaning that within a Gettext domain, a source string must be unique.
When representing a PO translation unit in XLIFF we cannot use the source string as
the value for the
We suggest the following approach for providing unique
It is however possible to use the PO format with logical ids, though this approach is not
much used. To support this, filters may add an optional function (specified
by a command-line flag or similar) to use For example: msgid "HELLO_WORLD" msgstr "Hello World!" would be mapped to: <trans-unit id="1" resname="HELLO_WORLD" xml:space="preserve"> <source>Hello World!</source> </trans-unit> After translation, the translated entry would be inserted as msgstr. For example:
<trans-unit id="1" resname="HELLO_WORLD" xml:space="preserve"> <source>Hello World!</source> <source>Hei verden!</source> </trans-unit>would be back-converted to PO as: msgid "HELLO_WORLD" msgstr "Hei Verden!"
Software messages commonly use escape sequences for
representing common control characters like newline ( For example, the following C source code fragment: printf("Please Enter the following Data:\n\
\t- First Name\n\
\t- Last Name\n");
would be represented in PO as: msgid "" "Please Enter the following Data:\n" "\t- First Name\n" "\t- Last Name\n" msgstr "" This fragment could be presented in XLIFF by preserving the escape sequences: <source>Please Enter the following Data:\n\t- First Name\n\ \t- First Name\n\t- Last Name\n</source>
which could be further enhanced by encapsulating escape characters in
<source>Please Enter the following Data:<ph id='1' ctype='lb'>\n</ph>\ <ph id='2' ctype='x-ht'>\t</ph>- First Name<ph id='3' ctype='lb'>\n</ph>\ <ph id='4' ctype='x-ht'>\t</ph>- First Name<ph id='5' ctype='lb'>\n</ph>\ </source> Or, the filters could replace escape sequences with the intended characters: <source>Please Enter the following Data: - First Name - Last Name </source>
The recommended approach, as also depicted in the table below, is as follows:
In addition, characters in a PO file that are not supported by the XML specification
(For example Vertical Tabulator VT -
Table 3. Handling of Common Escape Sequences
The
When extracting data from PO files, filters should use the
POT files are automatically generated by the Gettext tools, and is nothing but a simple string table containing the extracted translation units. POTs are much simpler than POs, which are modified by humans and contain additional meta-data (Translator comments, Header information). If PO is not used in the localisation process, it would in many situations be more feasable to convert directly from POT to XLIFF, and not use language-specific PO files at all in the localisation process. When converting from POT, the header can be ignored, as the header stored in POT is simply a skeleton header. When back-converting to PO, the filter can insert the neccecary PO header elements (MIME elements and optionally plural forms definitions), providing all data needed to produce the language specific MO files.
When plural translation units exist in the POT file, it is important to note
that it is impossible to send off a language neutral XLIFF file to translators.
Filters need to insert the correct number of
<?xml version="1.0" ?>
<xliff version="1.1" xmlns="urn:oasis:names:tc:xliff:document:1.1">
<file original="filename.po" source-language="en-US" datatype="po">
<body>
<trans-unit>
Each PO file maps to one XLIFF
The XLIFF may encapsulate the meta-data from the PO header in a
The XLIFF There are two recommended approaches to handling the PO header in XLIFF: Leaving the header out of the XLIFF file, or treating the header as a translation unit. Both approaches are described below. The information contained in the PO header is not needed in the localisation process, and can be left out of the XLIFF file. When converting POT files, it is possible to completely ignore the PO header, as described in Section 3.6, “Extracting from POT files”.
This approach involves storing the whole PO header as a XLIFF
For example: # French Translation for MyApplication. # Copyright (C) 2005 John Developer # This file is distributed under the same license as the MyApp package. # John Developer <john@example.com>, 2005. # Joe Translator <joe@example.com>, 2005. #, fuzzy msgid "" msgstr "" "Project-Id-Version: MyApp 1.0\n" "Report-Msgid-Bugs-To: MyApp List <myapp-list@example.com>\n" "POT-Creation-Date: 2005-04-27 13:15+0900\n" "PO-Revision-Date: 2005-04-27 13:45+0900\n" "Last-Translator: Joe Translator <joe@example.com>\n" "Language-Team: French Team <fr-list@example.com>\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" "Plural-Forms: nplurals=2; plural=(n!=1);\n" "X-Generator: KBabel 1.9\n" would be mapped to: <trans-unit id="1" restype="x-gettext-domain-header" approved="no" xml:space="preserve"> <source>Project-Id-Version: MyApp 1.0 Report-Msgid-Bugs-To: MyApp List <myapp-list@example.com> POT-Creation-Date: 2005-04-27 13:15+0900 PO-Revision-Date: 2005-04-27 13:45+0900 Last-Translator: Joe Translator <joe@example.com> Language-Team: French Team <fr-list@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plural-Forms: nplurals=2; plural=(n!=1) X-Generator: KBabel 1.9 </source> <target>Project-Id-Version: MyApp 1.0 Report-Msgid-Bugs-To: MyApp List <myapp-list@example.com> POT-Creation-Date: 2005-04-27 13:15+0900 PO-Revision-Date: 2005-04-27 13:45+0900 Last-Translator: Joe Translator <joe@example.com> Language-Team: French Team <fr-list@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plural-Forms: nplurals=2; plural=(n!=1) X-Generator: KBabel 1.9 </target> <note from="po-translator">Copyright (C) 2005 John Developer This file is distributed under the same license as the MyApp package. John Developer <john@example.com>, 2005. Joe Translator <joe@example.com>, 2005.</note> </trans-unit>
The content of the PO header can hardly be seen as translatable data, hence this approach is not fully faithful to the XLIFF specification. However, this approach is recommended as a "lesser-of-evils" approach in that it allows translators to modify PO header information - which is neccecary in many Gettext based localisation processes.
Each PO entry maps to a XLIFF For example: msgid "hello world" msgstr "hei verden" would be mapped to: <trans-unit id="1" xml:space="preserve"> <source>hello world</source> <target>hei verden</target> </trans-unit>
Each plural PO entry maps to a XLIFF For example: msgid "%d file deleted" msgid_plural "%d files deleted" msgstr[0] "%d fil slettet" msgstr[1] "%d filer slettet" would be mapped to: <group restype="x-gettext-plurals">
<trans-unit id="1[0]" xml:space="preserve">
<source>%d file deleted</source>
<target>%d fil slettet</target>
</trans-unit>
<trans-unit id="1[1]" xml:space="preserve">
<source>%d files deleted</source>
<target>%d filer slettet</target>
</trans-unit>
</group>
When the target language has more than two plural forms, the plural source
( For example: msgid "untranslated-singular" msgid_plural "untranslated-plural" msgstr[0] "translated-form-0" msgstr[1] "translated-form-0" ... msgstr[n] "translated-form-n" would be mapped to: <group restype="x-gettext-plurals">
<trans-unit id="1[0]" xml:space="preserve">
<source>untranslated-singular</source>
<target>translated-form-0</target>
</trans-unit>
<trans-unit id="1[1]" xml:space="preserve">
<source>untranslated-plural</source>
<target>translated-form-1</target>
</trans-unit>
...
<trans-unit id="1[n]" xml:space="preserve">
<source>untranslated-plural</source>
<target>translated-form-n</target>
</trans-unit>
</group>
When only one form exists for the target language (For example Japanese, Chinese, Korean),
the plural group should include a second For example: msgid "untranslated-singular" msgid_plural "untranslated-plural" msgstr[0] "translated-form-0" would be mapped to: <group restype="x-gettext-plurals">
<trans-unit id="1[0]" xml:space="preserve">
<source>untranslated-singular</source>
<target>translated-form-0</target>
</trans-unit>
<trans-unit id="1[1]" xml:space="preserve" translate="no">
<source>untranslated-plural</source>
</trans-unit>
</group>
It is important to be aware of the implications of plural forms when extracting data from language neutral POT files, as described in Section 3.6, “Extracting from POT files”.
Translator comments in PO have the same function as
It is possible to map each translator comment to a For example: # This is a comment that # goes over multiple lines msgid "hello world" msgstr "" could be mapped to: <trans-unit id="1"> <source>hello world</source> <note from="po-translator">This is comment that goes over multiple lines</note> </trans-unit>
Optionally, translator comments can be mapped to # This is a comment that # goes over multiple lines msgid "hello world" msgstr "" could be mapped to: <trans-unit id="1">
<source>hello world</source>
<context-group name="po-entry" purpose="information">
<context type="x-po-trancomment">This is comment that
goes over multiple lines</context>
</context-group>
</trans-unit>
It is up to the individual filter implementer to decide which approach (if not both) to use. Extracted comments in PO are comments extracted from source code, and provide a way for developers to add comments relating to a translation unit. They can be mapped to XLIFF in a similar fashion to Translator Comments.
It is possible to map each extracted comment to a For example: #. This is a comment that #. goes over multiple lines msgid "hello world" msgstr "" could be mapped to: <trans-unit id="1"> <source>hello world</source> <note from="developer">This is comment that goes over multiple lines</note> </trans-unit>
Optionally, extracted comments can be mapped to For example: #. This is a comment that #. goes over multiple lines msgid "hello world" msgstr "" could be mapped to: <trans-unit id="1">
<source>hello world</source>
<context-group name="po-entry" purpose="information">
<context type="x-po-autocomment">This is comment that
goes over multiple lines</context>
</context-group>
</trans-unit>
As with Translator Comments, it is up to the individual filter implementer to decide which approach (if not both) to use.
Each reference is mapped to two
Each reference is in addition grouped in a For example: #: example.c:34 otherfile.c:233 msgid "hello world" msgstr "" would be mapped to: <trans-unit id="1">
<source>hello world</source>
<context-group name="po-reference" purpose="location">
<context type="sourcefile">example.c</context>
<context type="linenumber">34</context>
</context-group>
<context-group name="po-reference" purpose="location">
<context type="sourcefile">otherfile.c</context>
<context type="linenumber">233</context>
</context-group>
</trans-unit>
The For example: #, fuzzy msgid "Hello world" msgstr "" msgid "Hello world!" msgstr "Hei verden!" should be mapped to: <trans-unit id="1" approved="no"> <source>hello world</source> </trans-unit> <trans-unit id="2" approved="yes"> <source>Hello world!</source> <target>Hei Verden!</target> </trans-unit>
If the For example: msgid "Hello world" msgstr "" #, fuzzy msgid "Hello world!" msgstr "Hei verden!" should be mapped to: <trans-unit id="1" approved="no"> <source>hello world</source> </trans-unit> <trans-unit id="2" approved="no"> <source>Hello world!</source> <target state="needs-review-translation">Hei Verden!</target> </trans-unit>
When back-converting to PO, the
The
Note that it is possible, when back-converting to PO, to honour
the
For example: <trans-unit id="1" approved="yes"> <source>As prompted on the following screen, please enter the following details: - First Name - Last Name </source> <target>Venligst fyll inn følgende data når du kommer til neste skjermbilde: - Fornavn - Etternavn </target> </trans-unit> would when back-converted be formatted as: msgid "" "As prompted on the following screen, please enter the following details:\n" " - First Name\n" " - Last Name\n" msgstr "" "Venligst fyll inn følgende data når du kommer til neste skjermbilde:\n" " - Fornavn\n" " - Etternavn\n" in favour of word-wrapping similar to this: msgid "As prompted on the following screen, please enter " "the following details:\n" " - First Name\n" " - Last Name\n" msgstr "Venligst fyll inn følgende data når du kommer til " "neste skjermbilde:\n" " - Fornavn\n" " - Etternavn\n"
How the
The
This flag can be honoured by extracting parameters to
#, c-format msgid "Hello %s, your score is %d." msgstr "Hei %s, du har %d poeng."
Here the parameters <trans-unit id="1" approved="yes"> <source>Hello <ph id="1" ctype="x-c-param">%s</ph>, your score is <ph id="2" ctype="x-c-param">%d</ph>.</source> <target>Hei <ph id="1" ctype="x-c-param">%s</ph>, du har <ph id="2" ctype="x-c-param">%d</ph> poeng.</target> </trans-unit>
Table 4. Recommended
For some source formats special concideration is needed when reordering parameters. For example: Hello %s, your score is %d. If we here in the target language wanted to write: Score: %d. Name: %s we would have to specify the position of the parameters: Score is %2$d for %1$s
Most XLIFF editors does not provide a way for translators to
edit the content of the For example, in the following PO fragment: #, c-foramt msgid "Hello %s, your score is %d." msgstr "" the extraction filter could insert neccesary ordering-tags when converting to XLIFF: <trans-unit id="1" approved="no"> <source>Hello <ph id="1" ctype="x-c-param">%1$s</ph>, your score is <ph id="2" ctype="x-c-param">%2$d</ph>.</source> </trans-unit> The translator could then safely re-order the parameters: <trans-unit id="1" approved="yes"> <source>Hello <ph id="1" ctype="x-c-param">%1$s</ph>, your score is <ph id="2" ctype="x-c-param">%2$d</ph>.</source> <target>Score is <ph id="2" ctype="x-c-param">%2$d</ph> for <ph id="1" ctype="x-c-param">%1$s</ph>.</target> </trans-unit> and the back converted PO file would then become: #, c-foramt msgid "Hello %s, your score is %d." msgstr "Score is %2$d for %1$s"
It is recommended to implement support for extracting parameters only if support for parameter re-ordering is also implemented.
If multiple domains are present in a PO file, it is recommended to group each
domain in a domain "domain_1" msgid "hello world" msgstr "hei verden" domain "domain_2" msgid "hello world" msgstr "hei verden" should be mapped to: <group restype="x-gettext-domain" resname="domain_1">
<trans-unit id="1">
<source>hello world</source>
<target>hei verden</target>
</trans-unit>
</group>
<group restype="x-gettext-domain" resname="domain_2">
<trans-unit id="2">
<source>hello world</source>
<target>hei verden</target>
</trans-unit>
</group>
In many cases a domain is not specified for the first translation
units of a PO file (They are said to belong to the default domain 'messages'). It is not
recommended to not group these translation units, but rather have them as children of the
msgid "hello world" msgstr "hei verden" domain "domain_2" msgid "hello world" msgstr "hei verden" should be mapped to: <trans-unit id="1">
<source>hello world</source>
<target>hei verden</target>
</trans-unit>
<group restype="x-gettext-domain" resname="domain_2">
<trans-unit id="2">
<source>hello world</source>
<target>hei verden</target>
</trans-unit>
</group>
[GNU Gettext] The GNU Gettext Manual http://www.gnu.org/software/gettext/manual [OASIS] Organization for the Advancement of Structured Information Standards Web site. [RFC 3066] RFC 3066 Tags for the Identification of Languages . IETF (Internet Engineering Task Force), Jan 2001. [XML 1.0] Extensible Markup Language (XML) 1.0 (Third Edition) . W3C (World Wide Web Consortium), Feb 2004 [XLIFF] XLIFF 1.1 Specification . OASIS XLIFF Technical Committee, October 2003. [XLIFF Tools] The XLIFF Tools Project http://xliff-tools.freedesktop.org/ | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||