Re: T2 Retry with Delivery Receipt

[Martin W Sachs <mwsachs@us.ibm.com>]

Chris said:  "However, I disagree that simply because the
software is not single-vendor sourced that we cannot state a
requirement that it adopt reliable store-and-forward with
equivalent semantics to what is stated for ebXML RM."

Chris, you missed my point.  I was arguing that precisely because the
software is NOT single-vendor sourced, we MUST state a reliability
requirement on the IMs.  (Violent agreement.)

Regards,
Marty

*************************************************************************************

Martin W. Sachs
IBM T. J. Watson Research Center
P. O. B. 704
Yorktown Hts, NY 10598
914-784-7287;  IBM tie line 863-7287
Notes address:  Martin W Sachs/Watson/IBM
Internet address:  mwsachs @ us.ibm.com
*************************************************************************************



Christopher Ferris <chris.ferris@sun.com> on 09/13/2001 11:48:33 AM

To:   David Fischer <david@drummondgroup.com>
cc:   ebXML Msg <ebxml-msg@lists.oasis-open.org>
Subject:  Re: T2 Retry with  Delivery Receipt



David,

Please see my comments below.

Cheers,

Chris

David Fischer wrote:
>
> I would like to reiterate my position.
>
> Requirement:
> ===========
> The ability to Retry from From Party MSH to To Party MSH on lack of a
Delivery
> Receipt is a fundamental functional requirement.  This does not exclude
the use
> of hop-to-hop reliable messaging.

Where is this requirement stated? Why didn't it get raised before?

>
> Rationale:
> =========
> The assertion that the summation of hop-wise reliable messaging equals
> end-to-end reliable messaging is an unproven assumption which can easily
be
> shown to be false for ebXML.

How so? Please demonstrate how it can "easily be shown to be false"?

>
> There have been many references to MQseries as an example of how to
accomplish
> Reliable Messaging, and to the lack of end-to-end acknowledgements in
MQseries.
> However, there are significant difference between MQseries and ebXML.  If
I may
> quote Marty:
>
>     IBM MQSeries is a stack of middleware that includes
>     much more function that just getting messages on and
>     off the wire. Since for MQSeries reliable messaging
>     purposes, each node on the path has the whole stack,
>     it is only necessary to postulate that all the nodes are
>     MQSeries, which implies the full set of function. With
>     ebXML, we cannot make silent assumptions about what
>     is above the MSH, especially with an intermediary. We
>     must make explicit statements about what must be there
>     for reliable messaging to work. . .

I agree that we must make explicit statements to clarify what the
responsibilities of an intermediary are w/r/t OnceAndOnlyOnce
deliverySemantics. However, I disagree that simply because the
software is not single-vendor sourced that we cannot state a
requirement that it adopt reliable store-and-forward with
equivalent semantics to what is stated for ebXML RM.

>
> We simply do not have the infrastructure or the proprietary requirements
to
> implement an MQseries-like system -- nor can we.  Any comparison between
ebXML
> Reliable Messaging and MQseries is simply not applicable.  Even if we
could make
> such comparisons, it will be some time before there is an ebXML
infrastructure
> sufficient to make any assumptions about the reliability of an entire
path.

So, you simply want to off-load all of the responsibility for everything
between From Party and To Party to the From Party MSH? Not practical.
We are specifying the infrastructure that you claim we don't have. We are
not defining proprietary requirements, nor do we need them. We can
establish
requirements and specify how these requirements are represented on the
wire.

>
> OTOH, end-to-end reliable messaging obviates the need for hop-wise
reliable
> messaging (but does not preclude it).  Chris has argued that end-to-end
Retries
> are a new requirement and not part of our original requirement list.  A
look at
> the Requirements document says otherwise.  There is no mention of
multi-hop and
> there is actually no mention of how Retries are to be accomplished at all
and
> only one mention of Retry in the entire document (Retry Count as a
parameter in
> the TSLA/CPA).  However, one could easily make the assumption that
Reliable
> Messaging tacitly assumes the function of Retries.  Very well, what does
the
> Requirements document say about Reliable Messaging?
>
> A scan of the document shows the implicit assumption of point-to-point
messaging
> only -- no middle hops.  Figures 1, 2, 3, 5 show:
>
>                  Sender                                      Receiver
>     Application <==> MSH <==> Internet <==> MSH <==> Application
>
> This implicit assumption of Sender to Receiver only is carried throughout
the
> document.  Nowhere does it show anything akin to an Intermediary Hop.

This does not necessarily reflect From and To Party, but two adjacent
compliant ebXML MSH nodes.

>
> The Requirements document does state that the receiving Party might be an
> intermediary agent but it makes no provision for multi-hop (an
intermediary does
> not automatically mean multi-hop).  The entire Requirements document
makes the
> assumption that Messaging is from the Sending Party to the Receiving
Party
> through some nebulous cloud (labeled as Internet).  Reliable Messaging is
from
> the Sender to the Recipient regardless of what might be in the
intervening
> cloud, or black box as we frequently called it.  Reliable Messaging never
> terminates inside the cloud/black box.  What happens in that black box is
of no
> concern to us nor do we even need to acknowledge its existence.  This was
the
> original axiom of multi-hop (although it seems some have forgotten).  The
> addition of multi-hop is actually the new requirement Chris speaks of.

No mention has ever been made that RM "terminates" at an intermediary.
The deliverySemantics is a QOS that is for the message, not an individual
hop.
Because we cannot require that all exchanges between adjacent nodes in a
message path will use the ebXML RM protocol, it could not be stated to be
an end-to-end protocol, but the QOS semantics most certainly ARE end-to-end
with the receiving MSH intermediary node assuming full responsibility to
store-and-forward, in a reliable manner the message to the next node in the
message path. How this is achieved/implemented is not our problem.

>
> How then shall we reconcile multi-hop to the Requirements document?
There is
> only one satisfactory answer -- add multi-hop in such a way that there is
no
> change in any other requirement in the document.  This means Reliable
Messaging,
> while it may ALSO be applied to Intermediary hops, cannot change the
fundamental
> functionality implicit in Reliable Messaging from end-to-end.  This
requires the
> ability to perform end-to-end RM functions (retries, duplicate detection,
> acknowledgements, failure notification) regardless of what may be going
on in
> the black box.  Everything that is added for multi-hop must be orthogonal
to
> basic end-to-end functionality prescribed by the Requirements document.
>
> What about the Technical reasons for end-to-end Retries?  There may be a
variety
> of temporary reasons for a failure.  Ostensibly, Reliable Messaging
guarantees
> either a successful send or a Delivery Failure Notification back to the
original
> sender.  Such a guarantee is easily shown to be impossible.  Simple
network
> failures can exceed HTTP retry timeouts and neither a forward send nor a
failure
> notification will be sent.  The quick answer in this case would be that
the
> local administrator will be notified and a phone call would be made.
However,
> even if such information (phone number or eMail address) were available
(it
> isn't), intermediary (VAN) operators with thousands of messages per hour
flowing
> through their systems from almost as many sources, will simply not
perform such
> tasks.  Intermediary Reliable Messaging is easily shown to be unreliable
just
> from common network outages.  As has been said numerous times, lack of
> notification cannot be relied upon as either a success or a failure.
Failure
> can never be proved.  Even a Failure message may not be construed as a
failure
> since it itself is based upon the lack of an Acknowledgement and thus an
unknown
> status.  There is never any proof of failure.  Conversely, lack of
failure is
> not a proof of success.  Only success may be proven.  The only proof of
success
> is an end-to-end acknowledgement (we erroneously call Delivery Receipt).
Must
> we then conclude that Intermediary Reliable Messaging is inherently
fallacious
> and thus only end-to-end Reliable Message is needed?  Yes! -- from the
point of
> view of the ends.  However, it can easily be shown that Intermediary
nodes just
> as much need their own end-to-end acknowledgements for their own
operations --
> which from the point of view of the ends looks like Intermediary Reliable
> Messaging.
>
> What happens if we don't provide end-to-end Retries?  If the Sender
requests a
> Delivery Receipt (end-to-end Acknowledgement) but does not receive it
then,
> regardless of the receipt of an intermediate acknowledgement, the Sender
must
> assume some form of failure.  The sending MSH must immediately report to
the
> application or to a human element that there was a failure.  The initial
human
> response would be to resend!!!  The second normal response would be a
phone call

Possibly, but not until after whatever was broken was fixed.

> to the receiving administrator (who will tell the sender to try again).
If
> there is no verification that the message was received, then the next
response
> would be to try to determine/correct the problem and resend.  Even if the
> message arrived and is verified by the receiver, a resend may still be
required
> to get the needed Delivery Receipt (NRR).  In *all* cases, the answer is
to

Not if the DR is delivered reliably.

> resend.  This resend MUST use the original MessageId for fear that the
wayward
> message will eventually be delivered or that the receiving administrator
is
> incorrect (especially if this is a financial transaction).  This means
that
> under ALL circumstances, the ability to resend from end-to-end is
REQUIRED and
> the ability for intermediaries to pass such a resend is necessary
functionality.

Again, the complexity that this adds (intermediaries are inherently
complex)
especially for processing intermediaries is prohibitive.

>
> What about the CPA parameters of Retries and RetryInterval?  In the case
of
> multi-hop, there will actually be two CPAs -- one with the end (CPA1) and
one
> with the next hop (CPA2).  The Retries to the first hop will be governed
by
> Retries & RetryInterval in CPA2.  What do Retries & RetryInterval in CPA1
mean?

This was not the design. Unfortunately, because we did not have adequate
time
to address intermediaries in CPP/A for v1.0, this is a little unclear. IMO,
we have a ways to go.

> Of course, these govern Retries with the end!  All the mechanisms needed
for
> end-to-end retries already exist -- no change is required in the
specification.

This is ludicrous. You cannot have end-to-end retries on top of
adjacent-node
retries without some new mechanism to indicate to the adjacent intermediary
that despite the fact that it has already seen this message, it should
process it again w/r/t routing, but NOT w/r/t any other processing that
it might normally undertake when it first received the message and
acknowledged
it. It means that there needs to be additional MSH processing to determine
if it *may* have previously seen the message. Based upon this decision, it
either
does process the message or it partially processes the message.

There are two sets of retries that the MSH must manage. We have to
reconsider
all of section 10 to support this manner of behaviour. You simply cannot
say that
no changes to the spec are required.

> The only problem is that the addition of multi-hop interferes with
end-to-end
> retries (duplicates) which, as we have seen, is a fundamental functional
> requirement under all circumstances when a Delivery Receipt is requested
but not
> received.

You're asking for retries on top of retries. What happens when the
end-to-end
retries are exhausted and there is still no delivery receipt? Do we add
retries
of retries of retries? What happens when they fail? Do we add yet another
layer?

Why not focus on what you perceive as an omission in the spec, that an
intermediary
has certain obligations w/r/t reliable delivery. Let's address that by
adding
text that fully sets out what the responsibilities of an intermediary are
not only w/r/t RM but w/r/t routing and any other oddities of an
intermediaries
role that is clearly distinct from that of an endpoint.

I'd like to focus on the specific use case that you cited in the call,
where
an MSH uses an EDI/INT gateway. Is there an ebXML MSH at the To Party or do
they
simply have an EDI/INT server?

     MSHA -> IMSHGW -> EDI/INTGW -> EDI/INTB

In this case, how does the ebXML delivery receipt get generated? IMO, the
EDI/INT Gateway has a responsibility to ensure that the message is safely
delivered. How it does this is not the perview of our specification.
However,
that doesn't obviate the responsibility that the gateway intermediary node
assumes.

>
> The question is then not whether to allow end-to-end Retries but rather
how to
> accomplish this since it MUST be allowed.  The automation of those
retries (or
> not) is not really important and can actually be left as an
implementation
> detail.
>
> Regards,
>
> David Fischer
> Drummond Group.
>
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>