Re: [ws-rx] AS need for ordered delivery?

[Tom Rutt <tom@coastin.com>]

Duane Nickull wrote:

>Further to my point, even if the AD receives all messages in order, this still doesn't mean it will process them in order.  
>
>The main question is "how far should a reliable messaging protocol go to try to ensure reliable exchanges 'can' happen"?  I think declarative mannerisms are our only option.
>
>Look forward to comments. 
>  
>
The current draft of ws-rm, has the following statement in the 
definitions of the DA:
"
There are four basic delivery assurances that endpoints can provide:
- AtMostOnce Messages will be delivered at most once without duplication 
or an error will be raised on at least one endpoint.  It is possible 
that some messages in a sequence may not be delivered.

-AtLeastOnce Every message sent will be delivered or an error will be 
raised on at least one endpoint.  Some messages may be delivered more 
than once.

-ExactlyOnce Every message sent will be delivered without duplication or 
an error will be raised on at least one endpoint.  This delivery 
assurance is the logical "and" of the two prior delivery assurances.

-InOrder Messages will be delivered in the order that they were sent.  
This delivery assurance may be combined with any of the above delivery 
assurances.  It requires that the sequence observed by the ultimate 
receiver be non-decreasing.  It says nothing about duplications or 
omissions.
"

the first three DA have a requirment to raise an error if not met.  The 
in order DA does not have this
requirement.

This needs to be clarified, id the DA places a requirment on the 
contract between rmd and the destanation app to indicate error, then 
that should be part of the spec.  This can be phrased in terms of 
"primitives" without saying how it is implemented.

Tom Rutt

>D 
>
>-----Original Message-----
>From: Duane Nickull [mailto:dnickull@adobe.com] 
>Sent: Wednesday, October 26, 2005 1:47 PM
>Cc: ws-rx@lists.oasis-open.org
>Subject: RE: [ws-rx] AS need for ordered delivery?
>
>Jacques et al:
>
>Is it really true that it has to be "received" by the AD in the order in which it is sent and that your two conditions are required?  Why couldn't they be delivered out of order with the order sequence numbers intact and let the AD (any ultimate destination for that matter) reconstruct the original message batch and figure out which ones to process first?   I really think the issue of DA's is not relevant to our specification.  We specify a marker than allows any destination to understand the order of the messages and whether or not some are missing.  Let them figure it out.  Where does it stop?  Will we start putting ordering of the SQL statements in reliable messages to ensure certain SQL inserts are done in order?  I say not.
>
>I think the real goal we are trying to accomplish is to allow the RMD to reassemble an exact copy of the streams that the RMS transmitted and understand if it has missed anything (period).  Once it has done that, it can sort out what to do with it when it has satisfactorily ascertained whether or not it has an accurate stream.
>
>Look at TCP for comparison.  What we are trying to do at the WS level is conceptually similar to the patterns and methods specified in TCP.  TCP destinations have the task of "recreating an exact copy of the data stream generated at the source, at the destination".  This must encompass that the stream be re-created in the same order and with no gaps or duplicates.  The mechanisms used to accomplish this assign a "unique" sequence number to each byte of data at its source, and to sort the bytes at the destination according to the sequence number.  The sorting operation corrects any disordering.  An acknowledgement, timeout, and retransmission scheme corrects for data loss.  The uniqueness of the sequence number corrects for data duplication.
>
>Note that nowhere does it get into what TCP clients or destinations do with all of this. It is merely an underlying assumption that TCP nodes owners have taken the appropriate precautions to ensure that the benefits of TCP are not unceremoniously discarded by poor handling of content once the stream is recreated.  Accordingly, DA's between TCP nodes and what lies beyond are not really relevant to that specification.  If you sent an HTTP get() to a server and get nothing back or a timeout error, you may guess that there is a network layer error; if the TCP layer succeeds, and you get back an HTTP error code, you know the error is higher up.  TCP itself does not care about the HTTP error codes nor should it.  The communications are a layered stack with upper layers depending on lower ones.  Likewise, some critical business functionality that demands an ERP system, a data base and an accounting system all get 3 messages, in order is probably not something that the WS-RX layer should see and understand.  It is only concerned with the fact that the outgoing stream can be either 100% accurately reproduced at the destination OR an error (fault) is thrown to notify participants something did not work.
>
>I would argue that we should focus on ensuring WS-RX has all the mechanics to allow an exact copy of all bytes being transmitted by the RMS to be received and assembled correctly by the RMD OR, in the alternative, that the RMD can understand something went wrong and it is missing something.  Mechanisms, contracts etc between the RMD and AD are not something to be judged by a wire protocol, we just have to ensure that the protocol supports the things required (which it does).
>
>Comments?
>
>Duane
>
>
>
>________________________________________
>From: Jacques Durand [mailto:JDurand@us.fujitsu.com] 
>Sent: Wednesday, October 26, 2005 1:17 PM
>To: 'Marc Goodner'; ashok.malhotra@oracle.com; Gilbert Pilz; Duane Nickull; Anish Karmarkar
>Cc: ws-rx@lists.oasis-open.org
>Subject: RE: [ws-rx] AS need for ordered delivery?
>
>Fact is, regardless if some agreement is needed about the DA to be used, InOrder is one of those DAs that actually *always* requires *both* parties to do a bit more than just using the protocol, meaning both AS/RMS and RMD/AD contracts. Here is why:
>For a "message to be delivered in the order it was sent", two conditions are required: 
>1-the message must be transmitted (by RMS) in the order it was sent (by AS)  (or at least, must be "sequence-numbered" in this order )--> contract RMS/AS
>2- the message must be delivered  (to AD) in the order it was received (by RMD) --> contract RMD/AD. 
>So far the focus has been on the Destination side, which is the one doing most of the legwork to achieve this. But the Source side must also do its part in supporting InOrder. 
>Now I believe that a specification only concerned with protocol RMS-RMD does not have to worry about this, but a spec detailing the DAs will have to.
>Jacques 
>
>  
>


-- 
----------------------------------------------------
Tom Rutt	email: tom@coastin.com; trutt@us.fujitsu.com
Tel: +1 732 801 5744          Fax: +1 732 774 5133