Re: T2 Retry with Delivery Receipt

[Christopher Ferris <chris.ferris@sun.com>]

Agreed.

Martin W Sachs wrote:
> 
> Chris said:  "However, I disagree that simply because the
> software is not single-vendor sourced that we cannot state a
> requirement that it adopt reliable store-and-forward with
> equivalent semantics to what is stated for ebXML RM."
> 
> Chris, you missed my point.  I was arguing that precisely because the
> software is NOT single-vendor sourced, we MUST state a reliability
> requirement on the IMs.  (Violent agreement.)
> 
> Regards,
> Marty
> 
> *************************************************************************************
> 
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287;  IBM tie line 863-7287
> Notes address:  Martin W Sachs/Watson/IBM
> Internet address:  mwsachs @ us.ibm.com
> *************************************************************************************
> 
> Christopher Ferris <chris.ferris@sun.com> on 09/13/2001 11:48:33 AM
> 
> To:   David Fischer <david@drummondgroup.com>
> cc:   ebXML Msg <ebxml-msg@lists.oasis-open.org>
> Subject:  Re: T2 Retry with  Delivery Receipt
> 
> David,
> 
> Please see my comments below.
> 
> Cheers,
> 
> Chris
> 
> David Fischer wrote:
> >
> > I would like to reiterate my position.
> >
> > Requirement:
> > ===========
> > The ability to Retry from From Party MSH to To Party MSH on lack of a
> Delivery
> > Receipt is a fundamental functional requirement.  This does not exclude
> the use
> > of hop-to-hop reliable messaging.
> 
> Where is this requirement stated? Why didn't it get raised before?
> 
> >
> > Rationale:
> > =========
> > The assertion that the summation of hop-wise reliable messaging equals
> > end-to-end reliable messaging is an unproven assumption which can easily
> be
> > shown to be false for ebXML.
> 
> How so? Please demonstrate how it can "easily be shown to be false"?
> 
> >
> > There have been many references to MQseries as an example of how to
> accomplish
> > Reliable Messaging, and to the lack of end-to-end acknowledgements in
> MQseries.
> > However, there are significant difference between MQseries and ebXML.  If
> I may
> > quote Marty:
> >
> >     IBM MQSeries is a stack of middleware that includes
> >     much more function that just getting messages on and
> >     off the wire. Since for MQSeries reliable messaging
> >     purposes, each node on the path has the whole stack,
> >     it is only necessary to postulate that all the nodes are
> >     MQSeries, which implies the full set of function. With
> >     ebXML, we cannot make silent assumptions about what
> >     is above the MSH, especially with an intermediary. We
> >     must make explicit statements about what must be there
> >     for reliable messaging to work. . .
> 
> I agree that we must make explicit statements to clarify what the
> responsibilities of an intermediary are w/r/t OnceAndOnlyOnce
> deliverySemantics. However, I disagree that simply because the
> software is not single-vendor sourced that we cannot state a
> requirement that it adopt reliable store-and-forward with
> equivalent semantics to what is stated for ebXML RM.
> 
> >
> > We simply do not have the infrastructure or the proprietary requirements
> to
> > implement an MQseries-like system -- nor can we.  Any comparison between
> ebXML
> > Reliable Messaging and MQseries is simply not applicable.  Even if we
> could make
> > such comparisons, it will be some time before there is an ebXML
> infrastructure
> > sufficient to make any assumptions about the reliability of an entire
> path.
> 
> So, you simply want to off-load all of the responsibility for everything
> between From Party and To Party to the From Party MSH? Not practical.
> We are specifying the infrastructure that you claim we don't have. We are
> not defining proprietary requirements, nor do we need them. We can
> establish
> requirements and specify how these requirements are represented on the
> wire.
> 
> >
> > OTOH, end-to-end reliable messaging obviates the need for hop-wise
> reliable
> > messaging (but does not preclude it).  Chris has argued that end-to-end
> Retries
> > are a new requirement and not part of our original requirement list.  A
> look at
> > the Requirements document says otherwise.  There is no mention of
> multi-hop and
> > there is actually no mention of how Retries are to be accomplished at all
> and
> > only one mention of Retry in the entire document (Retry Count as a
> parameter in
> > the TSLA/CPA).  However, one could easily make the assumption that
> Reliable
> > Messaging tacitly assumes the function of Retries.  Very well, what does
> the
> > Requirements document say about Reliable Messaging?
> >
> > A scan of the document shows the implicit assumption of point-to-point
> messaging
> > only -- no middle hops.  Figures 1, 2, 3, 5 show:
> >
> >                  Sender                                      Receiver
> >     Application <==> MSH <==> Internet <==> MSH <==> Application
> >
> > This implicit assumption of Sender to Receiver only is carried throughout
> the
> > document.  Nowhere does it show anything akin to an Intermediary Hop.
> 
> This does not necessarily reflect From and To Party, but two adjacent
> compliant ebXML MSH nodes.
> 
> >
> > The Requirements document does state that the receiving Party might be an
> > intermediary agent but it makes no provision for multi-hop (an
> intermediary does
> > not automatically mean multi-hop).  The entire Requirements document
> makes the
> > assumption that Messaging is from the Sending Party to the Receiving
> Party
> > through some nebulous cloud (labeled as Internet).  Reliable Messaging is
> from
> > the Sender to the Recipient regardless of what might be in the
> intervening
> > cloud, or black box as we frequently called it.  Reliable Messaging never
> > terminates inside the cloud/black box.  What happens in that black box is
> of no
> > concern to us nor do we even need to acknowledge its existence.  This was
> the
> > original axiom of multi-hop (although it seems some have forgotten).  The
> > addition of multi-hop is actually the new requirement Chris speaks of.
> 
> No mention has ever been made that RM "terminates" at an intermediary.
> The deliverySemantics is a QOS that is for the message, not an individual
> hop.
> Because we cannot require that all exchanges between adjacent nodes in a
> message path will use the ebXML RM protocol, it could not be stated to be
> an end-to-end protocol, but the QOS semantics most certainly ARE end-to-end
> with the receiving MSH intermediary node assuming full responsibility to
> store-and-forward, in a reliable manner the message to the next node in the
> message path. How this is achieved/implemented is not our problem.
> 
> >
> > How then shall we reconcile multi-hop to the Requirements document?
> There is
> > only one satisfactory answer -- add multi-hop in such a way that there is
> no
> > change in any other requirement in the document.  This means Reliable
> Messaging,
> > while it may ALSO be applied to Intermediary hops, cannot change the
> fundamental
> > functionality implicit in Reliable Messaging from end-to-end.  This
> requires the
> > ability to perform end-to-end RM functions (retries, duplicate detection,
> > acknowledgements, failure notification) regardless of what may be going
> on in
> > the black box.  Everything that is added for multi-hop must be orthogonal
> to
> > basic end-to-end functionality prescribed by the Requirements document.
> >
> > What about the Technical reasons for end-to-end Retries?  There may be a
> variety
> > of temporary reasons for a failure.  Ostensibly, Reliable Messaging
> guarantees
> > either a successful send or a Delivery Failure Notification back to the
> original
> > sender.  Such a guarantee is easily shown to be impossible.  Simple
> network
> > failures can exceed HTTP retry timeouts and neither a forward send nor a
> failure
> > notification will be sent.  The quick answer in this case would be that
> the
> > local administrator will be notified and a phone call would be made.
> However,
> > even if such information (phone number or eMail address) were available
> (it
> > isn't), intermediary (VAN) operators with thousands of messages per hour
> flowing
> > through their systems from almost as many sources, will simply not
> perform such
> > tasks.  Intermediary Reliable Messaging is easily shown to be unreliable
> just
> > from common network outages.  As has been said numerous times, lack of
> > notification cannot be relied upon as either a success or a failure.
> Failure
> > can never be proved.  Even a Failure message may not be construed as a
> failure
> > since it itself is based upon the lack of an Acknowledgement and thus an
> unknown
> > status.  There is never any proof of failure.  Conversely, lack of
> failure is
> > not a proof of success.  Only success may be proven.  The only proof of
> success
> > is an end-to-end acknowledgement (we erroneously call Delivery Receipt).
> Must
> > we then conclude that Intermediary Reliable Messaging is inherently
> fallacious
> > and thus only end-to-end Reliable Message is needed?  Yes! -- from the
> point of
> > view of the ends.  However, it can easily be shown that Intermediary
> nodes just
> > as much need their own end-to-end acknowledgements for their own
> operations --
> > which from the point of view of the ends looks like Intermediary Reliable
> > Messaging.
> >
> > What happens if we don't provide end-to-end Retries?  If the Sender
> requests a
> > Delivery Receipt (end-to-end Acknowledgement) but does not receive it
> then,
> > regardless of the receipt of an intermediate acknowledgement, the Sender
> must
> > assume some form of failure.  The sending MSH must immediately report to
> the
> > application or to a human element that there was a failure.  The initial
> human
> > response would be to resend!!!  The second normal response would be a
> phone call
> 
> Possibly, but not until after whatever was broken was fixed.
> 
> > to the receiving administrator (who will tell the sender to try again).
> If
> > there is no verification that the message was received, then the next
> response
> > would be to try to determine/correct the problem and resend.  Even if the
> > message arrived and is verified by the receiver, a resend may still be
> required
> > to get the needed Delivery Receipt (NRR).  In *all* cases, the answer is
> to
> 
> Not if the DR is delivered reliably.
> 
> > resend.  This resend MUST use the original MessageId for fear that the
> wayward
> > message will eventually be delivered or that the receiving administrator
> is
> > incorrect (especially if this is a financial transaction).  This means
> that
> > under ALL circumstances, the ability to resend from end-to-end is
> REQUIRED and
> > the ability for intermediaries to pass such a resend is necessary
> functionality.
> 
> Again, the complexity that this adds (intermediaries are inherently
> complex)
> especially for processing intermediaries is prohibitive.
> 
> >
> > What about the CPA parameters of Retries and RetryInterval?  In the case
> of
> > multi-hop, there will actually be two CPAs -- one with the end (CPA1) and
> one
> > with the next hop (CPA2).  The Retries to the first hop will be governed
> by
> > Retries & RetryInterval in CPA2.  What do Retries & RetryInterval in CPA1
> mean?
> 
> This was not the design. Unfortunately, because we did not have adequate
> time
> to address intermediaries in CPP/A for v1.0, this is a little unclear. IMO,
> we have a ways to go.
> 
> > Of course, these govern Retries with the end!  All the mechanisms needed
> for
> > end-to-end retries already exist -- no change is required in the
> specification.
> 
> This is ludicrous. You cannot have end-to-end retries on top of
> adjacent-node
> retries without some new mechanism to indicate to the adjacent intermediary
> that despite the fact that it has already seen this message, it should
> process it again w/r/t routing, but NOT w/r/t any other processing that
> it might normally undertake when it first received the message and
> acknowledged
> it. It means that there needs to be additional MSH processing to determine
> if it *may* have previously seen the message. Based upon this decision, it
> either
> does process the message or it partially processes the message.
> 
> There are two sets of retries that the MSH must manage. We have to
> reconsider
> all of section 10 to support this manner of behaviour. You simply cannot
> say that
> no changes to the spec are required.
> 
> > The only problem is that the addition of multi-hop interferes with
> end-to-end
> > retries (duplicates) which, as we have seen, is a fundamental functional
> > requirement under all circumstances when a Delivery Receipt is requested
> but not
> > received.
> 
> You're asking for retries on top of retries. What happens when the
> end-to-end
> retries are exhausted and there is still no delivery receipt? Do we add
> retries
> of retries of retries? What happens when they fail? Do we add yet another
> layer?
> 
> Why not focus on what you perceive as an omission in the spec, that an
> intermediary
> has certain obligations w/r/t reliable delivery. Let's address that by
> adding
> text that fully sets out what the responsibilities of an intermediary are
> not only w/r/t RM but w/r/t routing and any other oddities of an
> intermediaries
> role that is clearly distinct from that of an endpoint.
> 
> I'd like to focus on the specific use case that you cited in the call,
> where
> an MSH uses an EDI/INT gateway. Is there an ebXML MSH at the To Party or do
> they
> simply have an EDI/INT server?
> 
>      MSHA -> IMSHGW -> EDI/INTGW -> EDI/INTB
> 
> In this case, how does the ebXML delivery receipt get generated? IMO, the
> EDI/INT Gateway has a responsibility to ensure that the message is safely
> delivered. How it does this is not the perview of our specification.
> However,
> that doesn't obviate the responsibility that the gateway intermediary node
> assumes.
> 
> >
> > The question is then not whether to allow end-to-end Retries but rather
> how to
> > accomplish this since it MUST be allowed.  The automation of those
> retries (or
> > not) is not really important and can actually be left as an
> implementation
> > detail.
> >
> > Regards,
> >
> > David Fischer
> > Drummond Group.
> >
> > ----------------------------------------------------------------
> > To subscribe or unsubscribe from this elist use the subscription
> > manager: <http://lists.oasis-open.org/ob/adm.pl>
> 
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>
begin:vcard 
n:Ferris;Christopher
tel;cell:508-667-0402
tel;work:781-442-3063
x-mozilla-html:FALSE
org:Sun Microsystems, Inc;XTC Advanced Development
adr:;;One Network Drive;Burlington;Ma;01803-0903;USA
version:2.1
email;internet:chris.ferris@east.sun.com
title:Senior Staff Engineer
fn:Christopher Ferris
end:vcard