Re: T2 - Intermediaries/Message Status Request

[Martin W Sachs <mwsachs@us.ibm.com>]

My replies below.

*************************************************************************************

Martin W. Sachs
IBM T. J. Watson Research Center
P. O. B. 704
Yorktown Hts, NY 10598
914-784-7287;  IBM tie line 863-7287
Notes address:  Martin W Sachs/Watson/IBM
Internet address:  mwsachs @ us.ibm.com
*************************************************************************************



christopher ferris <chris.ferris@east.sun.com> on 08/11/2001 10:02:50 AM

To:   Martin W Sachs/Watson/IBM@IBMUS
cc:   "Burdett, David" <david.burdett@commerceone.com>, "'David Smiley'"
      <dsmiley@mercator.com>, ebxml-msg@lists.oasis-open.org
Subject:  Re: T2 - Intermediaries/Message Status Request



Please see below.

Cheers,

Chris

Martin W Sachs wrote:
<snip/>
>
> MWS:  The post example is a good example of an intermediary which simply
> forwards the message.  In the case of the post office, there may well be
> multiple hops in the process.  However in the case of the post office,
> the reliable messaging algorithm is definitely end to end.  I send it
> certified or registered and the delivery receipt confirms that the
> letter was received by the destination party.

How frequently do you send a letter certified or registered?

MWS:  Not very often - only when I need reliable messaging (e.g. sending a
stock certificate for sale).

It costs more for starters. It isn't very practical, especially
if you're expecting something in return from the recipient in
the near future.

MWS:  This is a good case for not using reliable messaging most ofthe
time. If you are expecting something in return, and are willing to tolerate
occasionally re-sending a request that will be processed as a new request,
you don't need reliable messaging and shouldn't pay the cost of it.

You send you bill payments via the post service, and you get
a new statement in about two weeks which would indicate whether
your payment was processed.

MWS:  You don't need reliable messaging for this.

Most people and most businesses don't send mail registered
or certified unless there's either something compelling
that warrants the additional cost. Maybe submission of a bid
or a time-sensitive or legal document.

MWS:  Precisely.

My point is that most of the time, you don't need a positive
ack from the ultimate recipient to achieve end-to-end reliability.

MWS:  When I don't need the positive ACK, I don't need reliable
messaging.

Consider MQSeries. The MQSeries infrastructure doesn't do end-to-end
acknowledgments within its own infrastructure. It is for the most part
only concerned with the next hop.

It *does* provide an optional application API that the application
can use to tell the sending application (not the MQSeries infrastructure)
that the message was received. But this isn't part of the reliable
messageing exactlyOnce delivery semantics that MQSeries offers.

MWS:  We really need John Ibbotson for this but he is on vacation
for most of August.  My (non-expert) understanding of MQ is that its
persistent
queues do the whole job and obviate the need for end to end ACKs or
delivery failure notification. MQ only works if both ends have the MQ
middleware. Its approach is probably too rich for ebXML.
MQ, itself, is proprietary so isn't a good candidate for a new transport
binding.

>
> >>>In the discussion of acks, delivery receipts, reliable messaging and
> floods, no mention is made of the use of Message Status Requests.<<<
>
> Definitely yes, see my other recent email. Greate minds think alike ;)
>
> David
>
> -----Original Message-----
> From: David Smiley [mailto:dsmiley@mercator.com]
> Sent: Thursday, August 09, 2001 3:07 PM
> To: 'Martin W Sachs'; Burdett, David
> Cc: ebxml-msg@lists.oasis-open.org
> Subject: T2 - Intermediaries/Message Status Request
>
> Being relatively new to the ebXML world, but learning more every day, I
> have
> been hesitant to jump in where those even more experienced may fear to
> tread, but here I go...
>
> I have extracted a relevant paragraph from the large body of the "T2
> SyncReply and ReliableMessagingMethod in
> QualityOfServiceI nfo" thread.
>
> <Martin W Sachs>
> The problem about this use case is that the intermediary is neither a
> participant in the business process nor strictly a
> pass through.  I suspect that means that it does participate in the
> business process though "not much".
> </Martin W Sachs>
>
> From a business perspective, please help me understand how any
intermediary
> is not a participant in the business process? If they are only a "pass
> through", why are they used? If they are involved, but "not much", is
that
> like being "a little pregnant" ;-)? It seems to me that any party which
> handles my business transactions, intelligently, persistently or
otherwise,
> is part of the business process.
>
> In the discussion of acks, delivery receipts, reliable messaging and
> floods,
> no mention is made of the use of Message Status Requests. I realize that
> this capability is optional in the implementation of a Message Service
> Handler, but wouldn't it be useful in some of the scenarios mentioned? In
> lieu of always resending a message whose delivery status is unknown, use
of
> Message Status Requests could reduce the number of large messages that
are
> resent unnecessarily. This may result in a very high tide, but floods may
> be
> avoided :-O.
>
> TIA for helping me understand this more thoroughly.
>
> David Smiley
> Director of Standards
> Mercator
>
> -----Original Message-----
> From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
> Sent: Thursday, August 09, 2001 4:47 PM
> To: Burdett, David
> Cc: ebxml-msg@lists.oasis-open.org
> Subject: RE: T2 SyncReply and ReliableMessagingMethod in
> QualityOfServiceI nfo
>
> David,
>
> In other words, if the Delivery Receipt does not arrive, the From party's
> MSH does not attempt to re-send the original message and may or may not
> inform the from application.  To me that says that the delivery receipt
is
> not part of reliable messaging. Failure to receive a delivery receipt is
> not a guarantee that the To party did not receive the original message.
> The From party is in as much doubt as to the status of the transaction as
> if reliable messaging had not been used.
>
> A phone call is not the answer here because it may not produce a timely
> resolution of the in-doubt status.  For example, the To party may only
> update its database overnight and thus customer service may not be able
to
> help until the next day.  If you submit a stock trade for immediate
> execution, you won't want to wait for the next day to decide whether you
> should resubmit the trade - and you will be nervous about resubmitting it
> right away because you don't want it executed twice. The purpose of
> reliable messaging is to let the From party know unambiguously, and in a
> timely manner, that the message either did or did not get to the To
party.
>
> Regards,
> Marty
>
>
****************************************************************************

>
> *********
>
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287;  IBM tie line 863-7287
> Notes address:  Martin W Sachs/Watson/IBM
> Internet address:  mwsachs @ us.ibm.com
>
****************************************************************************

>
> *********
>
> "Burdett, David" <david.burdett@commerceone.com> on 08/09/2001 04:24:24
PM
>
> To:   Martin W Sachs/Watson/IBM@IBMUS
> cc:   ebxml-msg@lists.oasis-open.org
> Subject:  RE: T2 SyncReply and ReliableMessagingMethod in
QualityOfServiceI
>                     nfo
>
> Marty said ...
>
> >>>Is the delivery receipt used by the From party's MSH in the reliable
> messaging algorithm?  If not, we still have the problem that the From
> party's MSH does not know whether the message got to the To party.<<<
>
> Yes and no. The ebXML RM spec provides two methods by which the From
Party
> MSH is notified of the status of the delivery of a message:
> 1. A negative ack - this is the "Delivery Failure" error message. This
> occurs whenever DeliverySemantics is OnceAndOnlyOnce and one of the MSHs
in
> the path (including the To Party MSH) cannot deliver the message to the
> final destination
> 2. A positive ack - this is the Delivery Receipt which, if its signed, is
> proof that the message was delivered.
>
> These can be used separately or together.
>
> So yes the Delivery Receipt is part of Reliable Messaging but there is no
> MSH action proscribed if it does not arrive, although probably it should
> notify the sending application.
>
> Does this answer the question?
>
> David
>
> -----Original Message-----
> From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
> Sent: Thursday, August 09, 2001 12:43 PM
> To: Burdett, David
> Cc: ebxml-msg@lists.oasis-open.org
> Subject: RE: T2 SyncReply and ReliableMessagingMethod in
> QualityOfServiceI nfo
>
> David,
>
> Comments in line.
>
> I assume that you intended to post to the list server, so I posted this
> response there.
>
> Regards,
> Marty
>
>
****************************************************************************

>
> *********
>
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287;  IBM tie line 863-7287
> Notes address:  Martin W Sachs/Watson/IBM
> Internet address:  mwsachs @ us.ibm.com
>
****************************************************************************

>
> *********
>
> "Burdett, David" <david.burdett@commerceone.com> on 08/09/2001 01:04:56
PM
>
> To:   Martin W Sachs/Watson/IBM@IBMUS
> cc:
> Subject:  RE: T2 SyncReply and ReliableMessagingMethod in
QualityOfServiceI
>                nfo
>
> Marty
>
> See comments inline.
>
> David
>
> -----Original Message-----
> From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
> Sent: Thursday, August 09, 2001 9:38 AM
> To: Burdett, David
> Subject: RE: T2 SyncReply and ReliableMessagingMethod in
> QualityOfServiceI nfo
>
> David,
>
> I agree that if the intermediary undertakes to guarantee forwarding of a
> message once it reaches the intermediary's persistent store, and
guarantees
> to return a delivery failure message, it might work. You do need to
> substitute SHALL for SHOULD in that sentence, however.
> [David B]I agree.
>
> The problem here is that when a delivery failure notification originates
in
> an external node, rather than in the From party's MSH, the delivery
failure
> notification is subject to the same losses that a normal message is
subject
> to, so the delivery failure notification itself has to be sent via
reliable
> messaging.
> [David B]I agree.
>
> I suspect that when this is looked at carefully, there may be
> opportuntities for things like delivery failure notifications of delivery
> failure notification messages, possibly multiple times around this loop.
> [David B]The spec says if a Delivery Failure message cannot be delivered
> reliably then the problem is solved by other means ... e.g. the phone.
See
> last paragraph of section 10.4
>
> MWS:  That's probably OK but we really should work on improving teh
> algorithm.
>
> There is also the question of the flood of ACKs from different
intermediate
> points that David Fischer referred to.
> [David B] I'm not sure where the flood is - apart from the flood of
> emails;). But the Acks only goes to the previous hop and the Delivery
> Receipt (if any) goes end to end.
>
> MWS:  Yes, this eliminates the flood (I was referring to one of David
> Fischer's
> postings).
>
> MWS:  Is the delivery receipt used by the From party's MSH in the
reliable
> messaging algorithm?  If not, we still have the problem that the From
> party's MSH
> does not know whether the message got to the To party.
>
> Regards,
> Marty
>
>
****************************************************************************

>
> *********
>
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287;  IBM tie line 863-7287
> Notes address:  Martin W Sachs/Watson/IBM
> Internet address:  mwsachs @ us.ibm.com
>
****************************************************************************

>
> *********
>
> "Burdett, David" <david.burdett@commerceone.com> on 08/09/2001 12:06:36
PM
>
> To:   Martin W Sachs/Watson/IBM@IBMUS
> cc:
> Subject:  RE: T2 SyncReply and ReliableMessagingMethod in
QualityOfServiceI
>           nfo
>
> >>However, I as I said earlier today,
> if the RM ACK is not end to end, there are
> opportunities for failures that won't be detected at the MSH level of the
> From party and will leave the From party in doubt
> as to whether the message got to its final destination.<<
>
> That's why you have the Delivery Failure error message which an
> intermediary
> (or even final) MSH should send back to the "From" MSH if they can't
> forward
> the message. Intermediaries are nothing special. If you are doing point
to
> point and the To Party MSH can't deliver a message to the Application
then
> the message delivery has failed.
>
> Best wishes.
>
> David
>
> -----Original Message-----
> From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
> Sent: Thursday, August 09, 2001 6:26 AM
> To: Burdett, David
> Subject: RE: T2 SyncReply and ReliableMessagingMethod in
> QualityOfServiceI nfo
>
> The problem about this use case is that the intermediary is neither a
> participant in the business process nor strictly a
> pass through.  I suspect that means that it does participate in the
> business process though "not much".
>
> We should support all these use cases.  However, I as I said earlier
today,
> if the RM ACK is not end to end, there are
> opportunities for failures that won't be detected at the MSH level of the
> From party and will leave the From party in doubt
> as to whether the message got to its final destination.  SMTP without RM
> works just as well for these cases.
>
> Regards,
> Marty
>
>
****************************************************************************

>
> *********
>
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287;  IBM tie line 863-7287
> Notes address:  Martin W Sachs/Watson/IBM
> Internet address:  mwsachs @ us.ibm.com
>
****************************************************************************

>
> *********
>
> "Burdett, David" <david.burdett@commerceone.com> on 08/08/2001 09:35:13
PM
>
> To:   "'SHIMAMURA Masayoshi (E-mail)'" <shima.masa@jp.fujitsu.com>, Scott
>       Hinkelman/Austin/IBM@IBMUS, Martin W Sachs/Watson/IBM@IBMUS,
>       "Christopher Ferris (E-mail)" <chris.ferris@east.sun.com>, "'David
>       Fischer'" <david@drummondgroup.com>, "Jim Hughes (E-mail)"
>       <jim_hughes@hp.com>
> cc:   "ebXML Messaging (E-mail)" <ebxml-msg@lists.oasis-open.org>
> Subject:  RE: T2 SyncReply and ReliableMessagingMethod in
QualityOfServiceI
>       nfo
>
> Masayosi/Scott/Marty/Chris/David F
>
> Phew! This is quite a thread which I have just caught up on. I also
> apologise for the length of this reply but to try and come to some type
of
> resolution I have:
> 1. Presented a specific use case involving an intermediary
> 2. Analyzed the use case to draw out some of the key issues that arise
> 3. Responded individual emails by extracting relevant items
>
> The bottom line is that, apart perhaps from some clarifications, I don't
> think the spec needs changing. Details below ...
>
> David
> --------------------------------------------------------
> USE CASE
> This use case describes the characteristics of two parties exchanging
> messages via an intermediary. Specifically:
> 1. There is one intermediary between the two parties.
> 2. When the intermediary receives a message from a "From Party" they do
not
> always forward it immediately to the "To Party"
> 3. The intermediary has separately agreed with each of the other two
> parties:
>    a) the messaging protocols that will be used between the intermediary
> and
> the party
>    b) that the intermediay will transform messaging protocols and
payloads
> if the From Party and To Party use different protocols.
> 4. Specifically the agreements are:
>    a) One party and the intermediary will use RosettaNet documents as
> payload with MQ Series without the ebXML Reliable Messaging protocols
>    b) The other party and the Intermediary use EDI Messages with ebXML
> messaging using SMTP and the ebXML Reliable Messaging Protocol
> 5. The intermediary does not take part in the business process between
the
> To and From Parties - i.e. the intermediary does not alter the semantic
> meaning of any of the data in the payload
> 6. The Application that is the ultimate destination at one of the parties
> runs in batch and therefore their MSH needs to store messages and deliver
> them to the application later.
>
> USE CASE ANALYSIS
> There is one intermediary
> Although there is only one intermediary the same principles will apply if
> there are many.
>
> The intermediary stores messages before forwarding
> This is a valid use case since it is possible that the final destination
of
> a message is off-line (e.g. they are a SME who only uses email) but they
> want to be able to let their customers also use HTTP.
>
> What this means is that there is benefit in the intermediary returning an
> "acknowledgment message" as the From Party can then turn off their retry
> process without waiting for the message to reach the final destination.
>
> The intermediary has separate agreements with each party
> What this means is that there two types of agreements in place:
> 1. Between the From or To Party and the Intermediary which covers how
> messages are sent but NOT the business process.
> 2. Between the From and To Party directly that cover the business process
> they will follow but not how messages are sent.
>
> As each party uses diferent transport protocols for reliable messaging,
one
> MQ Series and one the ebXML RM Protocol, you need to specify separately
for
> each hop which RM protocol to use. This is why reliableMessagingMethod is
> in
> the Via rather than the main message header.
>
> The application at the ultimate destination runs in batch
> If the rule is that application always provides the Delivery Receipt,
then
> the sender of the message might have to wait a long time to get a
response,
> even though the message has actually been received by the MSH running at
> the
> To Party. As the To Party is running both the MSH AND the Application it
is
> OK for the MSH to send the Delivery Receipt. If the Application is online
> and can send a receipt immediately after say it has checked the payload,
> then the MSH level Delivery Receipt is not necessary.
>
> Now for some comments on individual emails ...
>
> EMAIL THREAD
> The following are extracts from various emails on this topic with
responses
> to each one.
>
> David Fischer: Tue 8/7/2001 1:02 PM
> >>  We had long (heated) discussions about this in London and David (B)
> assured us that Acknowledgements were for intermediate hops only.
> Acknowledgement (sp) only goes back to the previous hop.  This is why it
is
> only in the Via.<<
> <DB>No. Acknowledgements are for Reliable Messaging using the ebXML RM
> only.
> If a message is being sent over MQ Series then an Acknowledgment message
is
> not needed (although it could be requested) as reliable delivery is
> provided
> in a different way. However whether you are using the ebXML RM protocol
or
> a
> proprietary protocol such as MQ Series, you still might want Non
> Repudiation
> of Receipt. This is why you need the Delivery Receipt separate from the
> Acknowledgement.</DB>
>
> Chris Ferris: Tue 8/7/2001 9:58 PM
> >>The following references cite discussion that clearly demonstrates that
> the team had felt that the delivery receipt is an application-level
> response
> [1] [2] [3] [4] and that we endeavored to separate the two so that it was
> clear that DR was not related to RM. <<
> ... and ...
> >>Again, it clearly states "To Party" which equates to "application"
which
> means that it is a business level signal. Nowhere in this section will
you
> find any discussion of reliable messaging because it is completely
> unrelated. A search of the Reliable Messaging section (10) in the spec
> turns
> up a whopping 0 (zero) occurances of the word DeliveryReceipt.<<
> <DB>I disagree. The Delivery Receipt is a MSH level response. For
example,
> in the use case where the To Party MSH stores a message before passing it
> to
> the application. Indeed, if the application receiving the message is an
old
> application it may not be able to generate the equivalent of a Delivery
> Receipt. Hence a Delivery Receipt produced by the MSH is useful. In
> addition, I don't think it is the MSH's responsibility to generate
messages
> business application responses.</DB>
>
> David Fischer: Tue 8/7/2001 6:03 PM
> (David B) I am also uncomfortable with having two parameters that imply
> essentially the same thing. For example if deliveryReceiptRequested or
> immediateAckRequested are both set to NONE, then a syncReplyMode of
> AcksOnly
> or AcksAndResponse would be inconsistent. We really ought be able to
avoid
> this sort of problem with the correct choice of parameters.
>
> (David F) I still agree. David Burdett may prefer AckRequested rather
than
> DeliveryRequested (David?).
> <DB>In this quote, the two parameters I was refering to as having similar
> functionality were the syncReplyMode and either or both of the
> deliveryReceiptRequested and intermediateAckRequested. The
> deliveryReceiptRequested and the intermediateAckRequested parameters have
> different meanings (see above).</DB>
>
> SHIMAMURA Masayoshi: Wed 8/8/2001 5:44 AM
> >>It means that if syncReply is set to False, the Reliable
Acknowledgement
> Message can not carry business level response. If so, at same time, the
> Reliable Acknowledgement Message can not carry deliveryReceipt element
when
> syncReply is set to False. Because deliveryReceipt element is always
> carried
> together business level response.<<
> <DB>As the Delivery Receipt is a MSH level response, it can be sent
> separately from the business level response.</DB>
>
> David Fischer: Wed 8/8/2001 8:56 AM
> >>How about section 9.2.1 & 9.2.2 which describes MSH level Ping & Pong?
> Yet the spec says the response is sent:
>      The message is then sent to the To Party.
> The words "To Party" & "From Party" in these sections are used to
describe
> the MSH, not an application.<<
> <DB>The To Party is BOTH the MSH AND the Application. The MSH is
equivalent
> to the mail room and the Application to the individual who opens and
> processes a letter. They are both part of the same party.</DB>
>
> Jim Hughes: Tue 8/7/2001 10:00 AM
> >>Maybe you get more "reliability" if an intermediate MSH node pair use
> MQseries, but I think it really confuses the issue when you allow the
> presence of intermediate nodes to affect the basic RM algorithm. I think
> that a node's use of RM should be quite distinct from the (higher layer?)
> RM
> between the endpoint MSHs.<<
> <DB>A node can usefully do its own RM processing especially if it stores
a
> message before forwarding it as can easily be the case. However hop by
hop
> RM does not mean that the sender knows that the final destination
received
> the message which is why you need a Delivery Receipt to let the original
> sender know.</DB>
>
> Scott Hinkelman: Tue 8/7/2001 10:17 AM
> >>I agree. RM is From <-> To. However, does this imply that if a CPA is
> used, and IMs are in the picture that multiparty (if an IM is a party)
CPA
> is required? Seems to me.<<
> <DB>Yes and no. In the exmple above, the intermediary is providing
> transformation services but does not "process" the payload in any
> conventional sense as they are not the final destination. For example you
> could organize it that all letters from French customers go to an
> outsourced
> translation service before they are sent to you. Is the translation
service
> an integral part of the business relationship between you and your French
> customers. I don't think so. Do your French customers need to know that
you
> have outsourced this part of your operation? Again, I don't think so. On
> the
> other hand there is definitely a business relationship between you and
the
> French translation service you are using which will need agreements in
> place.</DB>
>
> Scott Hinkelman: Wed 8/8/2001 7:22 AM
> >>(Marty). It is not at all clear that a multiparty CPA is always needed
to
> handle intermediaries.  It mostly depends on whether the intermediary
> particpates in the business process between the from and to party or acts
> as
> a pass-through. At the moment, the BPSS handles  a multiparty
collaboration
> as a set of
> (Scott) You are hitting on exactly the issue I brought up in [1]. I
believe
> we need to understand and clearify the functionality and responsibility
> differences between a Trading Partner and an IM in order to to resolve
much
> of this.<<
> <DB>Intermediaries CAN be pass throughs in that they do not process the
> payload. However intermediaries still have agreements with the parties
they
> work with. For example as an email user, you have an agreement with your
> ISP. However the people you exchange emails with don't need to know
> anything
> about your agreement with them yet the agreement still exists.</DB>
>
> Marty Sachs: Wed 8/8/2001 11:06 AM
> >>Certainly an adjacent pair of pass-through intermediaries can forward
> messages between themselves reliably "under the covers" but I have no
idea
> why they would do so since they have no skin in the outcome.  The only
ones
> the outcome matters to are the From and To parties.<<
> <DB>I disagree. They might have a lot of skin in the outcome. For example
> if
> an intermediary stores and forwards a message as a service then they will
> want to make sure that they do not lose messages they have received
before
> forwarding them.</DB>
>
> Let the discussion continue ... ;)
>
> Product Manager, xCBL, XML Standards
> Solution Strategy, Commerce One
> 4400 Rosewood Drive, Pleasanton, CA 94588, USA
> Tel/VMail: +1 (925) 520 4422; Cell: +1 (925) 216 7704
> mailto:david.burdett@commerceone.com; Web: http://www.commerceone.com
>
> ------------------------------------------------------------------
> To unsubscribe from this elist send a message with the single word
> "unsubscribe" in the body to: ebxml-msg-request@lists.oasis-open.org
>
> ------------------------------------------------------------------
> To unsubscribe from this elist send a message with the single word
> "unsubscribe" in the body to: ebxml-msg-request@lists.oasis-open.org