RE: [dita] Cross-Deliverable Links and Key Resolution

["Helfinstine, David" <dhelfinstine@ptc.com>]

Greetings,

I am still a little uncomfortable with some of the statements expressed below, but I came up with a couple statements that may / may not be correct to at least help me get closer to comfort. 

Peer maps with no keyscope (all maps defined with DITA 1.2 and prior). In this case "any" key not defined somewhere else "may" be defined in the peer map. So any message I think would be the same as before DITA 1.3, no need to say it "could" be in a peer map. (One thing is that if there is fallback markup in a key reference, then no message would be output in any case). I don't know if it would be good to note in a composition that "peer" maps were found in this case. If we were to keep to our definition from DITA 1.2 then we would not report this,  from the DITA 1.2 definition of peer in scope.

Peer maps with keyscopes (DITA 1.3) may need to have new behavior in that they need to be accessible (in some way, either in fact, or by proxy) for key space processing. Since this is "new" to DITA 1.3 I think that might be acceptable as a "should". If there is a keyscope then at least we could possibly be able to know if a message be issued if a keyref could not be resolved (because it would have a scope identifier). However, if a key is resolved by some other reference found in a map or submap with a scope-identifier but "might" have been defined in a peer, I don't think a message should be output. I see a lot of noise that could be generated.

Peer maps that have submaps that are local/peer also need to be processed, so this rule keeps going right? Peer maps may now have their own whole processing tree, with keys, more keyscopes, etc. Again, only if a keyscope is set on a peer map would we want to provide messages, etc. about any potential keyref not found. One thing that could happen is that the "peer" map could define a keyscope for the local map we are processing since they are peers of each other. That peer map would want to have references to our "local" map, probably with some sort of keyscope. I see possible circular references happening with this, but perhaps can be sorted out.

And, we are defining this for peer map processing of key spaces only, correct. Peer still means we don't normally process the peer map during normal processing of topics, correct? Or is that "scope" also expanded? It seems that if peer maps include key references to items in the peer map, then a new category of messages might now be "you do not have a topic defined in local map, but in a peer map". That may be OK also, but there are certainly some processing implications when you open this up.

These may be non-issues as this is still swirling in my mind, but thought I should put them down.

Thanks!

-Dave H.

Dave Helfinstine
dhelfinstine@ptc.com

-----Original Message-----
From: dita@lists.oasis-open.org [mailto:dita@lists.oasis-open.org] On Behalf Of Eliot Kimber
Sent: Monday, December 08, 2014 5:36 PM
To: Chris Nitchie; dita@lists.oasis-open.org
Subject: Re: [dita] Cross-Deliverable Links and Key Resolution

In thinking this through I've come to the conclusion that Chris' position is the correct one: there needs to always be a peer map that defines the keys defined in that map (or more abstractly, access to the peer map's root key space). If this is the case then normal key processing rules apply at all times and there's no need to have any special rules about key name matching.

I think the best way to express this in the specification is in terms of key space construction:

<p>
If the peer map's root key space is not available:
<ul>
<li>The processor MUST behave as though there was no peer map reference for the purpose of constructing the local key spaces. Processors SHOULD report locally-defined keys that could be overridden by keys defined in the peer map</li> <li>References to keys that would only be defined in the peer key space MUST be treated as undefined.</li> </ul> <note>The practical implication of this rule is that the owners of peer maps should provide at least a set of key definitions for those keys available in the peer map even if the peer map itself is not or cannot be made available (for example, for security reasons). If the owner of the peer map does not make the full map or key definitions available, the author of the linking map should create a set of key definitions to serve as a "proxy" for the peer map.
</note>
<note>When peer maps are referenced from the local map such that keys defined in the peer map could override keys with the same scope qualification defined in the local map, processors must have access to the peer map's root key space in order to accurately construct the local key spaces.
<p>
A locally-defined key can be overridden by a peer map if the locally-defined key is defined in a submap with a lower key definition precedence than the peer map. For example, in this map:
<codeblock><![CDATA[
<map>
  <mapref scope="peer"
    keyscope="book-02"
    href="../book-02/book-02.ditamap"
  />
  <mapref scope="local"
    keyscope="book-02"
    href="book-02-overrides.ditamap"
  />
  ...
</map>
]]>
</codeblock>
The peer map <filepath>book-02.ditamap</filepage> is included before the local submap  <filepath>book-02-overrides.ditamap</filepath>. This gives it higher import precedence, meaning that any keys it defines would override any keys defined in the <filepath>book-02-overrides.ditamap</filepath> map (which has the same scope name as the peer map). But a processor cannot determine if any keys defined in the <filepath>book-02-overrides.ditamap</filepath> map are overridden unless it knows what the key space for the peer map is.
<p>If the order of these two maps is reversed:
<codeblock><![CDATA[
<map>
<mapref scope="local"
  keyscope="book-02"
  href="book-02-overrides.ditamap"
/>
<mapref scope="peer"
  keyscope="book-02"
  href="../book-02/book-02.ditamap"
  />
...
</map>
]]>
</codeblock>
Then the local map <filepath>book-02-overrides.ditamap</filepath> has a higher key definition precedence and therefore any keys defined in that map would override the same keys defined in the peer map. In that case the peer map key space is not needed in order to determine the effective key definitions in the local map's key spaces.
</p>
</note>
</p>


Cheers,

E.

—————
Eliot Kimber, Owner
Contrext, LLC
http://contrext.com




On 12/8/14, 3:01 PM, "Chris Nitchie" <chris.nitchie@oberontech.com> wrote:

>That's not quite what I'm saying, and I feel pretty strongly that we
>shouldn't special-case peer map scope precedence. What I'm trying to get
>at is this:
>
>Given
>
><mapref keyscope="otherPub" scope="peer" href="pub.ditamap"/>
>
>And
>
><xref keyref="otherPub.someKey"/>
>
>Here is how I would want my processor to behave:
>
>* If pub.ditamap *can* be loaded, and the map does not contain a
>definition for 'someKey', processors should issue a WARNING or an ERROR
>stating that no definition for the referenced key could be found.
>* If pub.ditamap can be loaded and it contains a definition for someKey,
>huzzah, move on.
>* If pub.ditamap cannot be loaded, and no other definition for
>otherPub.someKey is found, processors should issue a MESSAGE or *maybe* a
>WARNING (*NOT* an ERROR) stating that the key could not be resolved, but
>may be defined in the peer map. (This, I think, is where we differ.)
>* If pub.ditamap cannot be loaded but the local map contains a definition
>for otherPub.someKey, huzzah, move on.
>
>Chris
>
>
>
>Chris Nitchie
>(734) 330-2978
>chris.nitchie@oberontech.com
>www.oberontech.com
> <http://www.oberontech.com/>
>Follow us:
> <https://www.facebook.com/oberontech>
> <https://twitter.com/oberontech>
> <http://www.linkedin.com/company/oberon-technologies>
> 
> 
>
>
>
>
>
>
>On 12/4/14, 4:10 PM, "Eliot Kimber" <ekimber@contrext.com> wrote:
>
>>I think I see what you're getting at.
>>
>>Applying the normal key definition precedence rules, if the map is as you
>>show it:
>>
>><mapref scope="peer" keyscope="otherPub" href="otherPub.ditamap"/>
>><keydef keys="otherPub.about" href="aboutThatOtherPub.dita"/>
>>
>>
>>Then if the peer map also defines the key "about" that will override the
>>local keydef (by normal key definition precedence) but if it doesn't
>>define it, then the local one would be effective.
>>
>>On the other hand, if the map was:
>>
>><keydef keys="otherPub.about" href="aboutThatOtherPub.dita"/>
>><mapref scope="peer" keyscope="otherPub" href="otherPub.ditamap"/>
>>
>>Then the first keydef clearly takes precedence over the peer scope,
>>again,
>>by normal key precedence rules.
>>
>>
>>That suggests that the rule has to be that peer scopes take precedence
>>over any later-defined key with the same scope qualifier. One way to say
>>this is that a peer scope implicitly defines *all possible* keys in that
>>scope. That resolves the ambiguity in a clear and simple way. We can also
>>say explicitly that processors may issue a warning in this case, because
>>it could be that the map author really did want to override that one key
>>and so they need to move it before the peer scope. That also suggests
>>that
>>as a matter of best practice, peer scope maprefs should go last in the
>>map.
>>
>>The only other option I can think of is actually requiring that the peer
>>maps be present at processing time and that was never our intent and
>>would
>>only be necessary to handle this case. It would set up a "Simon says"
>>requirement where you might be forced to create maps that declare
>>whatever
>>peer keys you decide to reference just so that the processor won't
>>complain, where you would otherwise not ever need the actual peer maps
>>because you're otherwise just using the key-to-address maps provided for
>>the target publications.
>>
>>Cheers,
>>
>>E.
>>
>>
>>—————
>>Eliot Kimber, Owner
>>Contrext, LLC
>>http://contrext.com
>>
>>
>>
>>
>>On 12/4/14, 2:38 PM, "Chris Nitchie" <chris.nitchie@oberontech.com>
>>wrote:
>>
>>>There is no way to tell whether a key reference is to a peer map simply
>>>by
>>>looking at it. You can tell if it specifies a scope qualifier that is
>>>mapped to a peer map, but that's not a 100% guarantee that the key
>>>should
>>>appear in that peer map.
>>>
>>><mapref scope="peer" keyscope="otherPub" href="otherPub.ditamap"/>
>>><keydef keys="otherPub.about" href="aboutThatOtherPub.dita"/>
>>>
>>>If, in a map containing the above, you see keyref="otherPub.intro", and
>>>otherPub.ditamap isn't available, there is *no way* for any processor to
>>>tell with 100% certainty whether it's a reference to a key that should
>>>be
>>>defined in the other publication, and so shouldn't be reported on, or a
>>>reference to a missing key definition in the 'local' keyscope structure,
>>>and so should be called out.
>>>
>>>So:
>>>
>>><quote>
>>>You can only know if a given key is or is not resolvable when you
>>>produce
>>>the final deliverable of the document doing the linking: between the
>>>time
>>>you author the link and when you produce the final form, anything could
>>>happen to the target document.
>>>Š
>>>For all these reasons, peer key references simply have to be ignored for
>>>the purpose of determining whether or not a key is resolvable as long as
>>>you're not producing the final-form deliverable.
>>></quote>
>>>
>>>At which point it's too late. There simply must be a way to validate
>>>keyrefs prior to final publication. If a keyref is not resolvable using
>>>the local key scope structure, but looks like it could refer to a key
>>>from
>>>an unavailable peer map, processors should say something. I don't think
>>>they should necessarily say the same thing they say for unambiguously
>>>unresolvable keyrefs (which is, I think, what the current language
>>>mandates), but they should say *something*.
>>>
>>>
>>>Chris
>>>
>>>
>>>Chris Nitchie
>>>(734) 330-2978
>>>chris.nitchie@oberontech.com
>>>www.oberontech.com
>>> <http://www.oberontech.com/>
>>>Follow us:
>>> <https://www.facebook.com/oberontech>
>>> <https://twitter.com/oberontech>
>>> <http://www.linkedin.com/company/oberon-technologies>
>>> 
>>> 
>>>
>>>
>>>
>>>
>>>
>>>
>>>On 12/4/14, 1:45 PM, "Eliot Kimber" <ekimber@contrext.com> wrote:
>>>
>>>>I think there may be some confusion about how I intended
>>>>cross-deliverable
>>>>references to be processed. This was captured in the discussion that
>>>>Michael and I had about how to implement cross-deliverable link
>>>>production
>>>>but since that concept didn't get included in the 1.3 spec I think most
>>>>people have not paid attention to it.
>>>>
>>>>I think the relevant aspect of cross-deliverable linking for this
>>>>discussion is that the facility as specified explicitly does not
>>>>require
>>>>that you know for sure that given peer key will actually be defined *at
>>>>the time you author the link*. The reason for this is that the
>>>>publications involved may be developed and produced asynchronously and
>>>>with little coordination. Thus the keys you want to link to may not in
>>>>fact have been literally defined at the time you author the links.
>>>>
>>>>You can only know if a given key is or is not resolvable when you
>>>>produce
>>>>the final deliverable of the document doing the linking: between the
>>>>time
>>>>you author the link and when you produce the final form, anything could
>>>>happen to the target document.
>>>>
>>>>In addition, if you use the generic key-based implementation approach
>>>>that
>>>>Michael and I developed, all references to peer keys become local key
>>>>references when you produce the final deliverable so normal key
>>>>resolution
>>>>rules apply during that final deliverable production process.
>>>>
>>>>For all these reasons, peer key references simply have to be ignored
>>>>for
>>>>the purpose of determining whether or not a key is resolvable as long
>>>>as
>>>>you're not producing the final-form deliverable.
>>>>
>>>>The reason that there is this distinction between production of the
>>>>final-form deliverable and any other processing you might be doing is
>>>>because resolving cross-deliverable links requires a multi-pass process
>>>>conceptually and that's how a lot of processors will implement it. In
>>>>particular, it is possible to have any amount of time elapse between
>>>>when
>>>>you do pass 1, as described below, and when you do pass 2: there is no
>>>>requirement that they be performed together in time. Therefore I think
>>>>we
>>>>can reasonably expect that most processors will actually reflect these
>>>>two
>>>>pass in their implementations.
>>>>
>>>>The passes are:
>>>>
>>>>Pass 1: Each publication involved in cross-deliverable linking is
>>>>processed once to determine, *for that publication*, what deliverable
>>>>anchors any keys become for that deliverable. This mapping of
>>>>keys-to-deliverable-addresses is saved for use in subsequent passes (it
>>>>was the details of how this data could be saved that Michael and I
>>>>discussed and arrived at the proposed interchange solution of using
>>>>intermediate key definitions).
>>>>
>>>>For example, if topic "topic-01.dita" is referenced by the topicref:
>>>>
>>>> <topicref keys="chapter-01" href="topic-01.dita">
>>>>
>>>>in the map and if for HTML output the result is HTML file
>>>>"chapter-01.html", then the deliverable-specific key-to-anchor mapping
>>>>would be "key 'chapter-01' maps to HTML file 'chapter-01.html'" for
>>>>this
>>>>deliverable. This mapping can be represented by a normal key definition
>>>>of
>>>>the form:
>>>>
>>>><keydef keys="chapter-01"
>>>>  href="../../publication-02/chapter-01.html"
>>>>  scope="external"
>>>>  format="html"
>>>>/>
>>>>
>>>>Pass 2: Each publication involved in cross-deliverable linking is
>>>>processed again, this time using the deliverable-specific key-to-anchor
>>>>mappings for each of the target publications to resolve any key
>>>>references
>>>>to those publications.
>>>>
>>>>Note that pass 1 does not *require* that any target peer maps be
>>>>available
>>>>because you're only concerned with keys within each publication (that
>>>>is,
>>>>generating that publication's key-to-anchor map).
>>>>
>>>>It is not until pass 2 that the processor has to be able to resolve the
>>>>cross-deliverable keys and that is the point at which failures can and
>>>>should be reported.
>>>>
>>>>Note also that there is an inherently loose coupling between these two
>>>>phases: in the general case you don't know when or if any given target
>>>>deliverable will itself be available and therefore you don't
>>>>necessarily
>>>>know during pass 1 processing if a given key will or won't actually be
>>>>resolvable when you go to do pass 2. You might have authored links to a
>>>>key that you expect will be defined but doesn't happen to be defined in
>>>>the target publication at authoring time. As long as that key is
>>>>defined
>>>>and resolvable when you do pass 2, it's all good.
>>>>
>>>>Thus, there can be processing contexts in which it is not known, and
>>>>doesn't need to be known, that a peer key reference can't be resolved,
>>>>namely the pass-1 processing for each publication.
>>>>
>>>>However, *if the peer maps are available*, processors certainly can
>>>>check
>>>>the key definitions if they choose to and report the issue. But how you
>>>>manage your related publications relative to each other and the
>>>>generation
>>>>of deliverables is entirely a business decision: you could impose very
>>>>tight controls or very loose controls, depending on what you need.
>>>>
>>>>The DITA-defined aspects of the process accommodate both loose and
>>>>tight
>>>>control.
>>>>
>>>>For that reason, we cannot state the rule for peer keys as "if you
>>>>can't
>>>>resolve the key it is treated as an unresolvable key" because there are
>>>>now valid processing contexts where you simply don't know if the key is
>>>>or
>>>>is not resolvable.
>>>>
>>>>I think the rule has to be stated in terms of producing final
>>>>deliverables: at that point, the normal unresolvable key rules should
>>>>apply. 
>>>>
>>>>But, there's more:
>>>>
>>>>The general mechanism Michael and I arrived at uses intermediate key
>>>>definitions as the way of capturing the key-to-anchor binding, as shown
>>>>above. 
>>>>
>>>>The basic idea is that in pass 1 you generate a set of key definitions
>>>>that reflect the key-to-anchor binding for the deliverable you're
>>>>creating. These keys are declared as scope="external" and with a format
>>>>reflecting the target deliverable (e.g., format="html", format="pdf" or
>>>>whatever it is).
>>>>
>>>>In pass 2, each publication that links to that deliverable literally
>>>>includes those keys before any locally-defined keys so that the
>>>>deliverable-specific keys take precedence.
>>>>
>>>>In this scenario, during pass 2 processing the key definitions are now
>>>>local to the publication making the cross-deliverable link, not peer,
>>>>and
>>>>so normal key processing rules apply: either the key is defined and
>>>>it's
>>>>all good or it's not and normal undefined key rules apply.
>>>>
>>>>Given this implementation approach, it should be clear that processors
>>>>should ignore peer key references, at least for the purposes of
>>>>applying
>>>>unresolvable key rules, because they can't know for sure if the key is
>>>>or
>>>>is not resolvable in the general case.
>>>>
>>>>However, DITA users can choose to impose a rule that all peer maps must
>>>>be
>>>>available during pass 1 processing and that they should reflect the
>>>>final
>>>>set of keys that will be available in that publication. This is the
>>>>"tightly controlled interlinked publication set" use case, e.g., what
>>>>might be provided by a CCMS that manages the authoring and publication
>>>>of
>>>>all the publications in a related set, enforcing specific business
>>>>rules
>>>>for release and publication. (This was the use case I typically had in
>>>>mind when thinking about this problem, e.g., the "all-knowing
>>>>publication
>>>>process manager".)
>>>> 
>>>>In that case processors can check the resolvability of peer key
>>>>references
>>>>early and report them or treat them as unresolvable during pass 1 (or
>>>>at
>>>>some appropriate workflow checkpoint where it is required that all
>>>>links
>>>>be resolvable). But that is an implementation and business rule choice
>>>>that is not inherent in the cross-deliverable link mechanism and that
>>>>cannot be mandated by the standard.
>>>>
>>>>Note that Michael had a completely different and equally-valid use case
>>>>in
>>>>mind: the "disconnected and lightly coordinated interlinked document
>>>>set",
>>>>where publications that link to each other are managed by different
>>>>groups
>>>>with very little direct coordination other than the interchange of the
>>>>key-to-anchor maps necessary to produce publications that link to other
>>>>publications.
>>>>
>>>>In the context of Michael's use case, it should be clear that trying to
>>>>enforce key resolvability during pass 1 is simply not generally useful
>>>>or,
>>>>in some cases, not possible, because you simply don't have the required
>>>>key-to-anchor mapping during initial authoring or maybe not until you
>>>>do
>>>>final deliverable generation for publication.
>>>>
>>>>In this disconnected case you might expect owners of documents to also
>>>>interchange maps that provide just the key definitions to which other
>>>>publications are allowed to link. In that case, early validation of key
>>>>references would be possible. But again, this level of coordination is
>>>>not
>>>>required by the facility as specified or intended.
>>>>
>>>>Cheers,
>>>>
>>>>E.
>>>>‹‹‹‹‹
>>>>Eliot Kimber, Owner
>>>>Contrext, LLC
>>>>http://contrext.com
>>>>
>>>>
>>>>
>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe from this mail list, you must leave the OASIS TC that
>>>>generates this mail.  Follow this link to all your TCs in OASIS at:
>>>>https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
>>>>
>>>
>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe from this mail list, you must leave the OASIS TC that
>>>generates this mail.  Follow this link to all your TCs in OASIS at:
>>>https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
>>>
>>>
>>
>>
>



---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php