recordPacking - A Proposal

["Hammond, Tony" <t.hammond@nature.com>]

Hi:

(Sorry for the length but needs to be said.)

> - Finally,  the orthogonality seems to me to make this parameter/element
> sematics overloaded and akward.

Well, I think we can all agree with that. But I think we need to understand
better the reasons why this has arisen. And then figure out what we're going
to do about it.

It seems to me that there *are* two distinct usages under the microscope:

    1. transport

    2. data access

The first usage ("transport") is what the original SRU 1.* use of
recordPacking sought to address. In effect it offered a means to escape the
XML data for transport purposes so that an XML data record could be snuck by
an XML processor. (Not clear why a CDATA section couldn't have been used for
this purpose.) Note that this is *not* data packing as is normally
understood. There is no actual packing involved - just XML escaping.

The second usage ("data access") is more in keeping with the named
operation: record packing. This relates to packing of record data into (and
out of) a schema - in this case the SRU/XML schema. This is a direct
consequence of SRU 2.0 taking on board alternate media types which are less
constrained than the canonical SRU/XML response. While we can define a
reasonable reference mappings of SRU/XML to these media types we have a real
world requirement to make the data more readily available to consumer apps.
(And if we allow the media type representations to vary too much from
standard SRU/XML we introduce destabilization of the protocol.)

So this would seem to suggest an impasse. But we need to make progress.

So let's do that.

    1. SRU 2.0 != SRU 1.*

    2. A "recordPacking" parameter suggests a value set of "packed" and
"unpacked" (or alterantely boolean equivalents)

    3. The "data access" issue is a data packing issue

    4. The "transport" issue is not a data packing issue

    5. SRU extension formats (RSS, etc) should align with the native format
(SRU) - or there should be reference registered versions that do align

    6. SRU extension formats are an integral part of the SRU 2.0 deal

On that basis I volunteer two alternate ways forward:

==
    A. Rename SRU 1.* "recordPacking" parameter to SRU 2.0
"recordXMLEscaping" with "xml", "string" values, and introduce new SRU 2.0
"recordPacking" parameter with "packed", "unpacked" values

    e.g.

    recordPacking = packed
    recordXMLEscaping = string

    Bottom line: We break the orthognality.

==
    B. Retain SRU 1.* "recordPacking" parameter but overload it so that it
provides hints to the server as to what actions to take viz-a-vis
"transport" issues and "data access" issues. We would not overspecify (way
too complex) but would leave server free to act.

    e.g.

    recordPacking - packed, string
    recordPacking - unpacked, xml

   Bottom line: We find a compromise solution.

==

Notes:

    * In my mind there is no contest but that SRU 2.0 should go with
Solution A: introduce new "recordXMLEscaping" parameter to handle the
restricted case of tunnelling XML record data.

    [[ And honestly, if this escaping was established for sole purpose of
passing back bad XML then the world will be a better place for not promoting
that activity! ]]

    * recordXMLEscaping could be generalized to recordEscaping but that
could affect the value set - e.g. do we add "json", etc in? Suggest we keep
this parameter purely for the SRU/XML case.

    * This is SRU 2.0 - a MAJOR revision. Taking on board alternate media
types comes at a price.

    * If we wimped out and went with Solution B there would still be
confusion as to what would happen with non-XML formats (e.g. JSON). Or would
we also throw a "json" value into the mix. And then what about other media
types? (Note that nature.com is next week deploying a "text/turtle" media
type. Would we also allow a "turtle" value?)

    * And if, horror of horrors, we go with the status quo then we are
propagating a badly named parameter for a limited use case which denies a
more general usability (and/or conformability) of open media types and
undermines credibility of the whole SRU effort. (Or something like that. ;)

Done wrong we risk ending up with another niche digital library protocol.
Done right we still have same background but we are at least facing forward
with respect to the Web and living up to the promise of SRU as a true ZING
thing.

Solution A is a bold step but IMHO is the correct way ahead.

Cheers,

Tony




********************************************************************************   
DISCLAIMER: This e-mail is confidential and should not be used by anyone who is
not the original intended recipient. If you have received this e-mail in error
please inform the sender and delete it from your mailbox or any other storage
mechanism. Neither Macmillan Publishers Limited nor any of its agents accept
liability for any statements made which are clearly the sender's own and not
expressly made on behalf of Macmillan Publishers Limited or one of its agents.
Please note that neither Macmillan Publishers Limited nor any of its agents
accept any responsibility for viruses that may be contained in this e-mail or
its attachments and it is your responsibility to scan the e-mail and 
attachments (if any). No contracts may be concluded on behalf of Macmillan 
Publishers Limited or its agents by means of e-mail communication. Macmillan 
Publishers Limited Registered in England and Wales with registered number 785998 
Registered Office Brunel Road, Houndmills, Basingstoke RG21 6XS   
********************************************************************************