RE: [docbook-tc] Review of DocBook transclusion proposal

["Rowland, Larry" <larry.rowland@hp.com>]

Sorry I missed the last meeting.  I am curious as to why idea of
on-the-fly conversion during transclusion was dropped while it is
still part of the assembly model.  This seems inconsistent and
will likely make things harder to explain (inconsistencies are
always harder to explain).  If the mechanism is already there
for assembly, wouldn't it be straightforward to use the same
mechanism for transclusion?  I think it really makes a more
compelling and useful model.

Regards,
Larry Rowland

-----Original Message-----
From: Jirka Kosek [mailto:jirka@kosek.cz] 
Sent: Thursday, September 16, 2010 8:14 AM
To: Grosso, Paul
Cc: DocBook Technical Committee
Subject: Re: [docbook-tc] Review of DocBook transclusion proposal

Grosso, Paul wrote:
> Per my action, I reviewed Jirka's transclusion proposal at
> http://lists.oasis-open.org/archives/docbook-tc/201007/msg00041.html

Many thanks, Paul. Please see my comments inline.

> I note that the document agrees that xinclude can handle most
> of the use cases except transclusion within attribute values
> (which the proposal also doesn't handle) and the issue of
> id-fixup.  I don't understand how the proposal handles UC-6
> any better than xinclude does.

Originally there was a plan to support arbitrary transformations on
content during transclusion (like on-the-fly conversion from DITA,
syntax highlighting, ...) -- this feature was rejected during the last
telcon, so yes the latest transclusion proposal is equivalent to
XInclude in this usec-case.

> So I wonder if it's worth defining a new mechanism rather 
> than using xinclude and requiring some id-fixup. 

Personally I think that disadvantage of XInclude is the very
inconvenient and verbose syntax. Compare:

<xi:include href="shared-texts.xml"
xpointer="xpath(id('product-name')/text())"/>

with

<ref definitionfile="shared-texts.xml" name="product-name"/>

Of course if you use some authoring tool which hides syntax then there
is no difference. But still there are plenty of people who prefer
directly type in angle brackets.

> Both for that reason and more generally, I wonder why the
> proposal says that xml:lang fixup should not occur, especially
> given that xml:base fixup does need to occur.

xml:lang fixup requires specifying xml:lang attribute on the top element
of each module (external shared chunk of XML) which is going to be
transcluded. Otherwise xml:lang="" will be automatically added which
means that language for content of this module is undefined which has
impact on the processing -- localization of gentext, hyphenation,
directionality of text, ...

I have been hit this before several times. I have used XIncludes for
content assembly and specified xml:lang="cs" only in the "master"
document. Then I was surprised why there was wrong gentext used on the
xincluded chapters (I haven't specified xml:lang on those chapters). I
think that for xml:lang much more natural behaviour is to inherit
language then to reset it.

> I assume the comments in the proposal about profiling
> (effectivity) really have nothing specific to do with the
> proposal; rather standard DocBook profiling would occur
> after transclusion, and this proposal isn't suggesting
> any changes with respect to profiling.  Correct me if
> I'm wrong.

Someone was mentioning DITA ambiguity in the similar case during telcon.

> ID fixup is the most complex part of things, but I'm
> pretty sure we don't need quite so many options.  With all 
> those options, we may not get as many implementations, and 
> it's more likely the implementations we do get may have bugs
> or incompatibilities.  I think we should just say that a 
> transclusion implementation needs to make all ids unique 
> and needs to fix up references, and leave most of the 
> details to the implementation.

I tend to agree, but there were requests for having manual control over
this process. If you think that the current proposal is too complex, we
can define two conformance levels -- "full transclusions" and "basic
transclusions". "basic" transclude process will have to support only
idfixup="auto" (which is default value).

> But fixing up references is still the complicated part.
> Clearly an id/idref link within a single piece of transcluded
> content should continue to work after id-uniquification.  As
> far as a link between two things that didn't start out (before
> transclusion) in the same piece of content, I'm not sure 
> what it means in all cases.  If someone has an idref in
> their document to an id in something they are transcluding
> and they transclude it more than once, what do they want?
> I'm not happy with either the near or global linkscope
> definition in the proposal, as they both seem like a hack.

Indeed, but I think that this hack meets 80/20 rule.

> The only reasonable interpretation is a one-to-many link,
> but I don't know if we want to get into that.  I'm not
> sure what to think here.

I don't think that users are ready to use one-to-many links. May be in
the next millennium ;-)

> Regarding:
> 
>  transclusions should be added into core DocBook. I don't
>  think that making them separate XML transclusion standard
>  is viable approach. Transclusion processor has to know which
>  attributes are of ID/IDREF type for each document type.
>  History shown that generic standards relying on access to
>  a schema are not successful.
> 
> Knowing which attributes are of ID/IDREF type is necessary
> for many things (e.g., XPath's id() function), and this is
> quite frequently done quite successfully for arbitrary 
> doctypes using schemas or some other method.

Please note that even DocBook XSL stylesheets are not relying on id()
function, they use xsl:key and key() to access IDs because access to
schema is not guaranteed. And this is even more true for DocBook V5.0
which uses RELAX NG as a primary schema. RELAX NG doesn't have anything
like <!DOCTYPE> or xsi:schemaLocation and it is expected that schema
information is transfered out-of-band using for example some external
schema mapping facility. It is very likely that not all tools in
processing toolchain will be able to access schema.

>  I do not think
> this is a good reason for hardwiring this proposal to DocBook,
> and it is possible we'd get more implementation support if we
> had a proposal that could add transclusion for other doctypes.
> It sounds to me like one just has to specify the element names
> used for the definitions, def, and ref (and maybe info) roles 
> (and their attributes), and we can define a doctype-agnostic
> transclusion feature.

It would be nice to use architectural forms for this ;-D

>  But maybe this is an implementation
> detail (i.e., some implementors will make it general but
> others won't) and doesn't really affect the DocBook TC
> proposal.

I think we should keep it simple and start with DocBook specific
solution. If it happens that it is successful it can be always
generalized for usage with other vocabularies.

				Jirka

-- 
------------------------------------------------------------------
  Jirka Kosek      e-mail: jirka@kosek.cz      http://xmlguru.cz
------------------------------------------------------------------
       Professional XML consulting and training services
  DocBook customization, custom XSLT/XSL-FO document processing
------------------------------------------------------------------
 OASIS DocBook TC member, W3C Invited Expert, ISO JTC1/SC34 member
------------------------------------------------------------------