Re: [search-ws] sru:recordPacking = unpacked ?

["Hammond, Tony" <t.hammond@nature.com>]

Hi Ralph:

> Sorry, but this doesn't make any sense.

It does. You will just need to bear with me.

> If you want to do OpenSearch,

I do. (But I also want SRU for its expressivity too.)

> why don't you just return an RSS or Atom response?

I am.

> If you do that, then
> you'll somehow do the transform from SRU to RSS/Atom on the server side
> and you'll be the only one to know how complex the logic was to find the
> necessary elements.

I am doing this server side.

And no, I disagree that I would be the "only one". Anyone familiar with a)
SRU, and b) the record schema being used would be able to make that
transform.

What I am simply proposing is that a server-side "unpacking" transform could
be provided as a convenience service. I just want to serve up the dish on
the plate, rather than leaving it in the can which is not very friendly.

I am operating on the basis that an SRU response can be mapped to other host
formats, e.g. RSS, ATOM, etc. (And this is already indicated by the Atom
extension model that has been provided.) I don't see any reason why SRU must
remain frozen in one XML format only, other than allowing for XSD validation
of the *complete* SRU response. If only XSD validation of the record data is
required then other carrier formats can equally host the SRU record data and
have an XSD validator run over that.

Given the above (hosting of SRU response), then I can present the SRU record
data within an OpenSearch document. The only problem is that to enhance the
ease of use and general utility of OpenSearch applications the record data
would ideally be unpacked and made immediately available.

**I see no reason why a search service should not return an SRU response
within a well known format (XML or otherwise), which is discoverable though
OpenSearch conventions, and that the data which is marshalled by the schema
be unmarshalled and allowed to percolate to the top for ease of use (or
"unpacked" as I have called it).**

One could take the view that any SRU vestiges are useless at that point and
the result should be "OpenSearch" only. I do not share that view. I think
the result could be regarded in a simple OpenSearch type view or from a more
rigorous SRU type view. (SRU responses allow for more information to be
passed back, e.g. Record schema, diagnostics, etc.) Note that even with the
record data unpacked, an SRU-aware application that was familiar with the
record schema being used could fairly readily "repack" the data.

In the earlier proposal I suggested a middle ground - "flattened" - which I
think should be dropped, so that there would only be two alternates:
"packed" and "unpacked".

In our nature.com OpenSearch service I would like to preserve the SRU
response elements in my return. I need though to make the search data
accessible. I will be doing that by unpacking the data and putting that at a
top level so that applications can make use of it. Free up the data. Surely
that's the point of the exercise? Or am I missing the point?

And I can do both - have data at the toplevel (OpenSearch style) and packed
away in an SRU record schema. I don't have any real problems with that
redundancy but that would effectively double the size of my responses. So I
am going to have to excise the SRU record data - or "unpack" it. I would
just have liked an indication at the SRU level that this is what has been
done.

You see. It does make sense. :)

Cheers,

Tony 

 


On 19/8/09 19:17, "LeVan,Ralph" <levan@oclc.org> wrote:

> Sorry, but this doesn't make any sense.  If you want to do OpenSearch,
> why don't you just return an RSS or Atom response?  If you do that, then
> you'll somehow do the transform from SRU to RSS/Atom on the server side
> and you'll be the only one to know how complex the logic was to find the
> necessary elements.
> 
> Ralph
> 
>> -----Original Message-----
>> From: Hammond, Tony [mailto:t.hammond@nature.com]
>> Sent: Wednesday, August 19, 2009 11:57 AM
>> To: search-ws@lists.oasis-open.org
>> Cc: Hammond,Tony
>> Subject: [search-ws] sru:recordPacking = unpacked ?
>> 
>> Hi:
>> 
>> I have a requirement for handling SRU record data to make it
> accessible
>> within an OpenSearch context. Essentially I need to "unpack" the
> record
>> data.
>> 
>> To explain this scenario I will use the example "Atom extension for
> SRU"
>> format [1] which is posted on the TC site.
>> 
>> This is a good start to the item level SRU mapping. So, for a result
> set
>> delivered in the "srw_dc" schema I might have something like:
>> 
>> <title>
>> <link>
>> <sru:recordSchema>...</sru:recordSchema>
>> <sru:recordPacking>...</sru:recordPacking>
>> <sru:recordData>
>>   <srw_dc>
>>     <dc:title>
>>     <dc:creator>
>>  </srw_dc>
>> </sru:recordData>
>> <sru:recordPosition>...</sru:recordPosition>
>> 
>> This is good from an SRU point of view but not so good from an
> OpenSearch
>> point of view as the search response elements ("dc:title",
> "dc:author") are
>> buried two levels down in the element hierarchy.
>> 
>> And this is significantly worse in my own use case of PAM, or PRISM
>> Aggregator Message, where there are two more wrapper elements
> intervening.
>> So I would have this:
>> 
>> <sru:recordData>
>>   <pam:message>
>>     <pam:article>
>>       <xhtml:header>
>>         <dc:title>
>>         <dc:creator>
>>      </xhtml:header>
>>    </pam:article>
>>  </ pam:message>
>> </sru:recordData>
>> 
>> From an OpenSearch point of view what I really need is this:
>> 
>> <title>
>> <link>
>> <dc:title>
>> <dc:creator>
>> <sru:recordSchema>...</sru:recordSchema>
>> <sru:recordPacking>...</sru:recordPacking>
>> <sru:recordData/>
>> <sru:recordPosition>...</sru:recordPosition>
>> 
>> That is, I need something akin to an "unpacked" option, so that my
> search
>> response elements are immediately available at the toplevel and not
> buried
>> within a hierarchy. The reason for this is not so much technical
> (although
>> it is that too) but is to do with mindshare. If it's too hard nobody
> will
>> bother and SRU is the loser. (SRU must accommodate OpenSearch, not the
>> other
>> way round.)
>> 
>> Note that we already have the request parameter "recordPacking" which
> takes
>> two values "xml" and "string". (And frankly, not doing very much IMHO.
> Badly
>> formed XML is already something outside of the SRU conventions -
> schema
>> then
>> amounts for nothing.)
>> 
>> (I believe this is the right parameter to focus on rather than
> introduce a
>> new parameter, e.g. "recordDataPacking".)
>> 
>> I wonder if it would be possible to either a) overload this parameter,
> or b)
>> substitute the application of this parameter, so that the following
> cases
>> could be requested:
>> 
>>     a) recordPacking = packed
>> 
>>     b) recordPacking = flattened
>> 
>>     c) recordPacking = unpacked
>> 
>> The first case a) would be the standard SRU offering, the second case
> would
>> allow for properties to bubble up to the <recordData> container
> element, and
>> the third case c) would allow for properties to escape the SRU
> response
>> entirely and be directly available to the host format (e.g. Atom in
> this
>> example).
>> 
>> Note that the "unpacked" option would not apply to native SRU response
> XML
>> which has nowhere to put this data - there is no "outside".
>> 
>> Note also that the "flattened" option may be too much to think about.
> It is
>> a logical possibility but may just complicate matters.
>> 
>> So, the three cases could look as folows:
>> 
>> a) recordPacking = packed
>> 
>> <title>
>> <link>
>> <sru:recordSchema>...</sru:recordSchema>
>> <sru:recordPacking>...</sru:recordPacking>
>> <sru:recordData>
>>   <srw_dc>
>>     <dc:title>
>>     <dc:creator>
>>  </srw_dc>
>> </sru:recordData>
>> <sru:recordPosition>...</sru:recordPosition>
>> 
>> 
>> b) recordPacking = flattened
>> 
>> title>
>> <link>
>> <sru:recordSchema>...</sru:recordSchema>
>> <sru:recordPacking>...</sru:recordPacking>
>> <sru:recordData>
>>   <dc:title>
>>   <dc:creator>
>> </sru:recordData>
>> <sru:recordPosition>...</sru:recordPosition>
>> 
>> c) recordPacking = unpacked
>> 
>> <title>
>> <link>
>> <dc:title>
>> <dc:creator>
>> <sru:recordSchema>...</sru:recordSchema>
>> <sru:recordPacking>...</sru:recordPacking>
>> <sru:recordData/>
>> <sru:recordPosition>...</sru:recordPosition>
>> 
>> 
>> Of course, we still have the existing values for "recordPacking"
> which, if
>> required still, could be accommodated by adopting a value list
> strategy,
>> similar to "sortKeys", e.g. we could have
>> 
>>     * recordPacking = xml
>> 
>>     * recordPacking = xml, packed
>> 
>>     * recordPacking = xml, flattened
>> 
>>     * recordPacking = xml, unpacked
>> 
>> and same for "string".
>> 
>> It could be argued that an "unpacked" record data is no longer a valid
> SRU
>> response, but I would counter that not all responses have recoprd
> data, e.g.
>> where diagnostics are provided. Also, even in an "unpacked" scenario
> with an
>> empty "recordData" element (or missing element) we have said something
> about
>> the data: that is was there but has been processed ("unpacked").
>> 
>> I would like to hear some responses to this proposal as I will very
> shortly
>> have to go ahead and implement search responses for the service we are
> about
>> to release. I would like to be as fully SRU compatible as possible but
> also
>> making it a breeze for OpenSearch users. I believe that the current
> proposal
>> (or something very much like it) would allow for both perspectives to
> be
>> maintained.
>> 
>> Cheers,
>> 
>> Tony
>> 
>> 
>> [1]
>> 
> http://www.oasis-open.org/committees/download.php/33408/atom-extension-f
> or-
>> s
>> ru.xml
>> [2]
>> 
> http://www.crossref.org/CrossTech/2009/07/opensearch_formats_for_review.
> ht
>> ml
>> 
>> 
>> 
> ************************************************************************
> ********
>> DISCLAIMER: This e-mail is confidential and should not be used by
> anyone who
>> is
>> not the original intended recipient. If you have received this e-mail
> in error
>> please inform the sender and delete it from your mailbox or any other
> storage
>> mechanism. Neither Macmillan Publishers Limited nor any of its agents
> accept
>> liability for any statements made which are clearly the sender's own
> and not
>> expressly made on behalf of Macmillan Publishers Limited or one of its
> agents.
>> Please note that neither Macmillan Publishers Limited nor any of its
> agents
>> accept any responsibility for viruses that may be contained in this
> e-mail or
>> its attachments and it is your responsibility to scan the e-mail and
>> attachments (if any). No contracts may be concluded on behalf of
> Macmillan
>> Publishers Limited or its agents by means of e-mail communication.
> Macmillan
>> Publishers Limited Registered in England and Wales with registered
> number
>> 785998
>> Registered Office Brunel Road, Houndmills, Basingstoke RG21 6XS
>> 
> ************************************************************************
> ********
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe from this mail list, you must leave the OASIS TC that
>> generates this mail.  Follow this link to all your TCs in OASIS at:
>> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
>> 
> 
> 


********************************************************************************   
DISCLAIMER: This e-mail is confidential and should not be used by anyone who is
not the original intended recipient. If you have received this e-mail in error
please inform the sender and delete it from your mailbox or any other storage
mechanism. Neither Macmillan Publishers Limited nor any of its agents accept
liability for any statements made which are clearly the sender's own and not
expressly made on behalf of Macmillan Publishers Limited or one of its agents.
Please note that neither Macmillan Publishers Limited nor any of its agents
accept any responsibility for viruses that may be contained in this e-mail or
its attachments and it is your responsibility to scan the e-mail and 
attachments (if any). No contracts may be concluded on behalf of Macmillan 
Publishers Limited or its agents by means of e-mail communication. Macmillan 
Publishers Limited Registered in England and Wales with registered number 785998 
Registered Office Brunel Road, Houndmills, Basingstoke RG21 6XS   
********************************************************************************