Re: T2 Retry with Delivery Receipt

[christopher ferris <chris.ferris@Sun.COM>]

David,

Please see below.

Cheers,

Chris

David Fischer wrote:
> 
> I haven't seen any discussion on the list from this question.  Does this mean
> everyone agrees there are valid use cases supporting end-to-end retries?

I don't agree that there has been a valid use case presented. 

Specifically, what we are concerned with are intermediary nodes
that are ebXML MSH nodes, not transport intermediary nodes such as SMTP
nodes along an SMTP message path between a From: and a To: email address.

e.g.

	MSHA->SMTP(local)->SMTP(x)->SMTP(y)->SMTP(To:<host>)->MSHB

The above is a *single hop* from the ebXML RM perspective. If the message
is either delayed, or "lost" somewhere between MSHA and MSHB, then the
ebXML RM protocol would kick in and there would be an automated retry
based on retryInterval/retries as detailed in the spec because MSHA
would not receive an Acknowledgment from MSHB. The same holds true for
the case where the Acknowledgment is lost on the return message path.
MSHA would automatically resend the message. MSHB would detect any
duplicates, based on MessageId and respond with the *same* Acknowledgment
that it sent in response to the original message it received.

If MSHA wants a DR for NRR from MSHB, it asks for this and would receive
it. I believe that this makes a strong case for the DR to be delivered
reliably, so that MSHA can be certain that it will be received.

In the use case where there is not an MSH at the To Party, as might be
the case where some manner of gateway is employed, then the MSH which 
acts as that gateway has a responsibility to ensure that the message is
reliably delivered. I don't believe that it is our responsibility to
provide specification language for how that is to be achieved.

e.g.

	MSHA->HTTP->MSHB||EDI/INT->EDI/INT(To Party)

In the above case, MSHB is the endpoint from the perspective of the
sending MSHA. MSHB would acknowledge the message from MSHA after it
had persisted the message, thus having assumed full responsibility for
delivering the message. How this is effected is outside the scope of
our specification. MSHB (or more correctly, the gateway "application"
at MSHB) can do whatever is necessary to ensure that the message is
delivered. It can resend the message all it likes as far as I'm concerned.
MSHA should not have to concern itself with these details since it
successfully transitioned the responsibility to reliably deliver the 
message to the node at MSHB.

In the case where there is a reverse gateway at the To Party, then
again, the ebXML RM protocol could be used to ensure that the To Party
receives one and only one message just as was the case for the transfer
between MSHA and MSHB. How you get a DR for NRR from the To Party
in the above case is also beyond our scope. It must be assumed that
somehow, the To Party generates some manner of EDI/INT equivalent to
the DR which the MSHB||EDI/INT gateway translates into an ebXML DR.
Again, we don't need to go there for the spec because it is outside
our scope.

e.g.

	MSHA->HTTP->MSHB||EDI/INT->EDI/INT||MSHC

In the above case, if the message were lost between MSHB and MSHC,
then the retries kick in, etc. thus ensuring that the message
is safely delivered. In this use case, EDI/INT is equivalent
to a transport such as HTTP (even if EDI/INT uses HTTP for its
transport).

We do not assume that MSHB||EDI/INT is unreliable. We assume that
it is reliable. I see no reason why we need to pursue the case where
it is irresponsible and may lose messages or not bother to make
any effort at ensuring that the message is reliably delivered
safely to the next MSH node. 

In the case where there IS no subsequent MSH node, then all bets
are off as far as I'm concerned. We need not concern ourselves
with this because it is outside our scope. We are and should be
solely concerned with ensuring that we have a reliable messaging
protocol that works effectively between MSH nodes that exchange
messages over an unreliable transport protocol such as HTTP, SMTP
or orange-juice cans and strings.

If you want to covver the case where some disaster at MSHB||EDI/INT
GW node results in loss of data, then MSHB needs to rollback to some
earlier state (one in which it has not seen the messages that it may have lost).
MSHA can resend messages which MSHB will treat as new, forwarding
them onto MSHC which would discard any duplicates as per the current
spec. Any messages which are resent in this manner, which MSHC has not
previously received would be processed accordingly. 

Does this satisfy you your end-to-end retry requirement? Note that
it doesn't involve any changes to the spec (IMO). If you or anyone
else wants to build in an automated retry on non-receipt of a DR,
you are free to do so. I disagree that it is something that the
specification needs to comment on. The retries could be manual
with equal effect (and that would also provide that MSHB had
restored itself to some stable state if one assumes that a phone
call or some other OOB communication is made to ensure that everything
is ready to roll). For that matter, MSHA may have a similar rollback
capability that it could invoke after consulting the To Party
(possibly using the Status inquiry MSH service) to determine
at which point the two parties need to resynchronize, etc.

Again, I have also repeatedly stated that a DR is not a requirement
for all messages in all cases. It may be that it is frequently used,
but in fact it may be mere window dressing to some. Some parties
might consider the expected "response" or follow-on message as 
the proof they require to "know" that the message they sent
was received. They may be satisfied with "if I get no business
response, then the business transaction is null and void". This
is likely to be something that the Business Server layer of software
would concern itself with, not the MSH. 

> 
> The way the spec is written now, single-hop, end-to-end retries work.  Multi-hop
> end-to-end retries do not work when RM is turned on (idempotence).  Can we now
> discuss what that will entail?
> 
> Regards,
> 
> David Fischer
> Drummond Group.
> 
> -----Original Message-----
> From: David Fischer [mailto:david@drummondgroup.com]
> Sent: Friday, September 14, 2001 8:46 AM
> To: Martin W Sachs; Christopher Ferris
> Cc: Dan Weinreb; ebxml-msg@lists.oasis-open.org
> Subject: RE: T2 Retry with Delivery Receipt
> 
> This all comes down to "Are end-to-end Retries REQUIRED"?  All the other things
> like automated retries, end-to-end RM, retry on DR, are secondary issues.
> 
> Under any delivery failure scenario, the ability to retry the original send is
> REQUIRED.  This might be automated or it might be manual.  It might come from
> the MSH or from the Application.  It might be now or after a fix.  No matter
> where or how, we MUST allow end-to-end Retries.
> 
> Can anyone disagree with this?
> 
> Regards,
> 
> David Fischer
> Drummond Group.
> 
> -----Original Message-----
> From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
> Sent: Friday, September 14, 2001 8:25 AM
> To: Christopher Ferris
> Cc: Dan Weinreb; david@drummondgroup.com; ebxml-msg@lists.oasis-open.org
> Subject: Re: T2 Retry with Delivery Receipt
> 
> Sure but it is an example of how ebxml end to end RM can work through
> unreliable IMs.
> 
> ********************************************************************************
> *****
> 
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287;  IBM tie line 863-7287
> Notes address:  Martin W Sachs/Watson/IBM
> Internet address:  mwsachs @ us.ibm.com
> ********************************************************************************
> *****
> 
> Christopher Ferris <chris.ferris@sun.com> on 09/13/2001 01:40:58 PM
> 
> To:   Martin W Sachs/Watson/IBM@IBMUS
> cc:   Dan Weinreb <dlw@exceloncorp.com>, david@drummondgroup.com,
>       ebxml-msg@lists.oasis-open.org
> Subject:  Re: T2 Retry with Delivery Receipt
> 
> Marty,
> 
> AN SMTP node is NOT an MSH node. It is not part of the equation.
> The MSH nodes that are communication via SMTP are the ones that
> adopt the RM protocol of retries in the absence of an Acknowledgment.
> The SMTP nodes are incidental.
> 
> Cheers,
> 
> Chris
> 
> Martin W Sachs wrote:
> >
> > Re:  "I think David's position is that we can't do that, because there
> are
> > hosts/entities out there that (a) must participate as ebXML MS IM's,
> > and (b) that are unreliable.  The question is whether there's a use
> > case demonstrating this."
> >
> > There is one major use case, which is SMTP.  SMTP intermediate nodes are
> > notoriously unreliable and only acknowledge to the previous node so a
> > sender has no idea whether the message got to its destination.  ebXML on
> > top of SMTP is one of the major reasons for having ebXML reliable
> messaging
> > and only end to end reliable messaging helps with SMTP.  I don't know if
> > there is a use case for ebXML unreliable intermediaries but if there is,
> > end to end RM is the answer.
> >
> > Regards,
> > Marty
> >
> ********************************************************************************
> *****
> 
> >
> > Martin W. Sachs
> > IBM T. J. Watson Research Center
> > P. O. B. 704
> > Yorktown Hts, NY 10598
> > 914-784-7287;  IBM tie line 863-7287
> > Notes address:  Martin W Sachs/Watson/IBM
> > Internet address:  mwsachs @ us.ibm.com
> >
> ********************************************************************************
> *****
> 
> >
> > Dan Weinreb <dlw@exceloncorp.com> on 09/13/2001 12:55:02 PM
> >
> > Please respond to Dan Weinreb <dlw@exceloncorp.com>
> >
> > To:   chris.ferris@sun.com
> > cc:   david@drummondgroup.com, ebxml-msg@lists.oasis-open.org
> > Subject:  Re: T2 Retry with  Delivery Receipt
> >
> >    Date: Thu, 13 Sep 2001 11:48:33 -0400
> >    From: Christopher Ferris <chris.ferris@sun.com>
> >
> >    > The only problem is that the addition of multi-hop interferes with
> > end-to-end
> >    > retries (duplicates) which, as we have seen, is a fundamental
> > functional
> >    > requirement under all circumstances when a Delivery Receipt is
> > requested but not
> >    > received.
> >
> >    You're asking for retries on top of retries. What happens when the
> > end-to-end
> >    retries are exhausted and there is still no delivery receipt? Do we
> add
> > retries
> >    of retries of retries? What happens when they fail? Do we add yet
> > another layer?
> >
> > What David is asking for is perfectly sensible *if* you your failure
> > model states that IM's are unreliable, e.g. that an IM might accept a
> > message, and then silently forget it.  In that case, the end-to-end
> > retries exist for a specific purpose: to harden the system against the
> > possibility of flaky IM's.  There would be no need to add another
> > layer unless there is some additional, distinct failure mode to be
> > taken care of.
> >
> >    Why not focus on what you perceive as an omission in the spec, that an
> > intermediary
> >    has certain obligations w/r/t reliable delivery. Let's address that by
> > adding
> >    text that fully sets out what the responsibilities of an intermediary
> > are
> >    not only w/r/t RM but w/r/t routing and any other oddities of an
> > intermediaries
> >    role that is clearly distinct from that of an endpoint.
> >
> > I think David's position is that we can't do that, because there are
> > hosts/entities out there that (a) must participate as ebXML MS IM's,
> > and (b) that are unreliable.  The question is whether there's a use
> > case demonstrating this.
> >
> >    I'd like to focus on the specific use case that you cited in the call,
> > where
> >    an MSH uses an EDI/INT gateway. Is there an ebXML MSH at the To Party
> or
> > do they
> >    simply have an EDI/INT server?
> >
> >         MSHA -> IMSHGW -> EDI/INTGW -> EDI/INTB
> >
> >    In this case, how does the ebXML delivery receipt get generated? IMO,
> > the
> >    EDI/INT Gateway has a responsibility to ensure that the message is
> > safely
> >    delivered. How it does this is not the perview of our specification.
> > However,
> >    that doesn't obviate the responsibility that the gateway intermediary
> > node
> >    assumes.
> >
> > I'd call this a protocol-translating gateway, not an ebXML MS IM at
> > all.  I agree that the gateway has to make sure that the message is
> > truly delivered, and then the gateway generates the DR.  It's the
> > job of the protcol-translating gateway to create the illusion that
> > the far end is really running ebXML MS.
> >
> > ----------------------------------------------------------------
> > To subscribe or unsubscribe from this elist use the subscription
> > manager: <http://lists.oasis-open.org/ob/adm.pl>
> >
> > ----------------------------------------------------------------
> > To subscribe or unsubscribe from this elist use the subscription
> > manager: <http://lists.oasis-open.org/ob/adm.pl>
> 
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>
> 
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>