RE: FW: T2, Proposed solution for ... Re: SyncReply andReliableMessagingMethod in QualityOfServiceInfo

[David Fischer <david@drummondgroup.com>]

Thanks Chris, idempotence goes after the IF, my mistake.

You said:

  We don't say whether or not the MSH is actually responsible for
  the packaging of the messages, we only say how they need to be
  packaged so as to ensure interoperability.

Not true.  Look at figure 6-1 and accompanying text.   Look at B.3.6.  The
packaging is applied by the MSH.  Signature must be applied after packaging so
it also must be in the MSH.  Encryption is always applied after signature so it
too must be applied by the MSH.  There may be a set of callable modules (helper
apps) which comprise the MSH -- no problem.  This whole set would then be called
the MSH.

There has been some talk towards defining a standard API to the MSH.  If we do
this then we must be very careful to define what functions are in the MSH.
Chris, if you want to do something different, that's fine as long as the message
structure is compatible with the specifications.  However, when we get around to
defining an API, your way won't work any more.

On Routing, there are two kinds.  First, end-to-end (single-hop) which is really
just transport and is definitely within the bounds of MSH.  Second, IM Routing
which is actually only a modification of the Via/TraceHeaderList and then
Transport.  We do not specify how this is done and frankly, I don't think it
works now.  If the IM Routing is an application then I must seriously question
whether this is an IM or it is an end.  Functions decrypt/verify
signature/encryption are within the bounds of the MSH (note MSH errors to
these).  Since it is unlikely that a true store/forward IM could do these
functions, it seems essential that IM routing be a part of the MSH and not an
application function.  (MSH helper app should be fine).

Regards,

David Fischer
Drummond Group.

-----Original Message-----
From: christopher ferris [mailto:chris.ferris@sun.com]
Sent: Monday, September 24, 2001 1:27 PM
To: ebXML Msg
Subject: Re: FW: T2, Proposed solution for ... Re: SyncReply and
ReliableMessagingMethod in QualityOfServiceInfo


David,

I don't believe that I am suggesting any change whatsoever.

The psuedo-code you cite below is nearly correct. The difference
that I see is whether the idempotency check is before the
IF you cite or afterwards. If we are going to specify that
something along the lines of RetryCount be appended to
the MessageId for purposes of performing an idempotency check
at an intermediary, then that function needs to be afterwards
(part of both 1) and 2) below, each using a different comparison).

I think that it needs to be said also that we are not specifying an
implementation, but a protocol and specified behaviour. If we were
developing an implementation specification, then clearly it would
be a very different document. We don't say anything about
what software entity actually applies or verifies the
digital signature, nor anything about what it might mean.
We only specify that if you want to apply one, you do it "this way"
so that there can be some chance of interoperability.

We don't say whether or not the MSH is actually responsible for
the packaging of the messages, we only say how they need to be
packaged so as to ensure interoperability.

We do impose certain constraints on the RM function, such as the
requirement that before the Acknowledgment is sent, the message
MUST be persisted and that the receiving software MUST be capable
of recovering from a failure such that messages that have not been
processed are processed as if there had been no failure. We just
don't say anything about *how* this miracle is pulled off by any
given implementation.

We say nothing about how the "application" interacts with the
MSH, nor how an MSH dispatches messages, nor how application
software is "bound to" a MSH such that the MSH can dispatch
messages to the correct application handler. We provide
Service & Action, but we don't necessarily say that
dispatching by the MSH is based on Service and Action. The MSH
could perform its dispatching of messages based on other
content such as CPAId or ConversationId. For that matter,
dispatching might be based on nothing other than the fact
that the message is an ebXML message (such as might be the case
for an intermediary node concerned solely with routing
or some other mundane function that is not specific to
the business end of the message).

As for routing, we don't say how some intermediate node
knows or figures out what URL maps to a given PartyId URI
or how it knows or determines what transport should be
used for the next hop. We are completely silent on the
matter as this is clearly not something that needs to be said
as far as the protocol and required behaviour is concerned.
This is strictly up to an implementer to choose as they
see fit.

As for the routing application, we (TR&P team) agreed in Tokyo, IIRC,
that the actual routing was not a function of the MSH but that there
was a "routing" application (which is very different than saying that
the application does routing). This decision was an important one as it
allowed us to move beyond a sticking point in our discussions
when we had to think about intermediary routing as part of the
MSH.

As I have said before, this doesn't mean that the routing function
might not be baked into the same software entity that includes the
MSH functionality. It might be just that, but technically, this isn't
part of the MSH and to be honest, it also meant that this was
not the only possible solution.

Cheers,

Chris

David Fischer wrote:
>
> We can't put all these functions in the application.  We are supposed to
support
> Transport, Routing and Packaging.
>
> What you are suggesting is pushing Routing & Packaging into the
application?!!!
> The application is just supposed to pass parameters and payload(s) to the MSH
> where the ebXML-MS headers are built, signed, encrypted and sent.  The other
end
> receives the message, parses the SOAP headers, performs Idempotency (RM), and
> then performs an IF:  1) if not the To Party then modify the packaging (Via,
> TraceHeaderList) and send to next-hop/end, 2) if the To Party, validate
> signatures/encryption and dispatch the payload(s) to the application based
upon
> Service/Action.  Look at figure 6-1.  We do Header Parsing & Header
Processing.
> We do RM.  We also do Security Services.  What I left out is Error Processing
at
> each stage of the MSH.
>
> The IF in the above paragraph was introduced by multi-hop.  If no multi-hop
then
> do 2.  Multi-hop does not appear in the figure (and let's NOT put it in).
>
> What you are suggesting reduces the MSH to just a message send & receive
> function.  Why bother?  You are suggesting a massive change in scope and
> functionality from the current specification.
>
> Regards,
>
> David Fischer
> Drummond Group.
>
> -----Original Message-----
> From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
> Sent: Friday, September 21, 2001 5:40 PM
> To: David Fischer
> Cc: Chris.Ferris@Sun.COM; ebXML Msg
> Subject: RE: FW: T2, Proposed solution for ... Re: SyncReply and
> ReliableMessagingMethod in QualityOfServiceInfo
>
> A few comments below.
>
> Regards,
> Marty
>
>
********************************************************************************
> *****
>
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287;  IBM tie line 863-7287
> Notes address:  Martin W Sachs/Watson/IBM
> Internet address:  mwsachs @ us.ibm.com
>
********************************************************************************
> *****
>
> David Fischer <david@drummondgroup.com> on 09/20/2001 05:45:07 PM
>
> To:   Chris.Ferris@Sun.COM, ebXML Msg <ebxml-msg@lists.oasis-open.org>
> cc:
> Subject:  RE: FW: T2, Proposed solution for ... Re: SyncReply and
>       ReliableMessagingMethod in QualityOfServiceInfo
>
> Thanks Chris, very good points.
>
> <df>comments in-line</df>
>
> Regards,
>
> David Fischer
> Drummond Group.
>
> -----Original Message-----
> From: Chris.Ferris@Sun.COM [mailto:Chris.Ferris@Sun.COM]
> Sent: Thursday, September 20, 2001 11:50 AM
> To: ebXML Msg
> Subject: Re: FW: T2, Proposed solution for ... Re: SyncReply and
> ReliableMessagingMethod in QualityOfServiceInfo
>
> David Fischer wrote:
> >
> > As Chris pointed out, we need to carefully consider this.  So, I am
> sending
> this
> > out again (slightly modified -- retryCount goes in Via).  Does anyone see
> any
> > problems with this or does anyone have an alternative proposal?
> >
> > Regards,
> >
> > David Fischer
> > Drummond Group.
> > ===============
> > Proposal:
> > A retryCount attribute on Via.  This parameter can only be set by the
> From
> Party
> > MSH and MUST start at zero.  The first retry, retryCount is set to one,
> etc.
>
> How does this work if the sending MSH knows nothing of intermediaries?
> Are you suggesting that the Via element is now REQUIRED on all messages?
>
> <df>Yes, this is a problem, not only for retryCount but for all parameters
> in
> Via.  I have asked this same question before, is Via always required?  If
> we are
> doing RM at all, then Via is required in order to pass the ackRequested and
> reliableMessagingMethod parameters.  I guess Via is REQUIRED for all RM
> messages -- regardless of retryCount.
> </df>
>
> >
> > Discussion:
> > Duplicate detection, as always, is detected by the To Party and From
> Party
> based
> > upon MessageId.  Duplicate detection by the Intermediary is based upon
> MessageId
> > plus retryCount.
>
> How does an intermediary distinguish itself as such? This seems to preclude
> an MSH "module" that can be used anywhere as there is now a distinction
> between
> how an intermediary MSH node treats idempotency checking versus how an
> endpoint
> does so.
>
> MWS:  Is it really a separate "module" or just an installation parameter.
> See
> next comment.
>
> MWS:  If we agree that ALL intermediary function (even simple forwarding)
> is above the level of the MSH (i.e. an "appication"), we might make more
> forward progress/
>
> <df>The IM must know that it is an IM in order to forward rather than
> process
> the payload.  If the IM is not the From Party or the To Party then it must
> be an
> IM.  This is not quite true for a mailroom situation (need to work on this
> --
> something like MTA routing tables?  or MX records?).
> </df>
>
> MWS:  Again, if all the IM function is an application process, this is not
> a problem
> and an MSH does not have to care if it is in an intermediary or an end
> node.
>
> >
> > For example (see attached flow chart):
> >
> > Let's say there are three intermediaries (nodes B, C, D).  A is the From
> and E
> > is the To.  A sends a message with MessageId of 1 and retryCount of 0
> (let's
> > call this ID 1.0).  We will assume two failures occur during send -- the
> message
> > makes it to D but has problems sending to E.  The Ack from C also fails
> going
> > back to B.  Since A did not get a DR within retryInterval, it decides* to
> retry
>
> retryInterval applies to the RM retry. You are now stating that the DR is
> the RM
> artifact, not the Acknowledgment. What happens when no DR is
> requested/required?
> Does the sending MSH wait for the response message?
>
> <df>Chris, you are right, almost.  There is a CPA to the first IM and a CPA
> with
> the end.  There is a retryInterval in each which corresponds to that
> interval.
> However, what if the From Party does not know about any IMs?  I think it
> would
> be wise for ends to always make retryInterval long and IMs to make
> retryInterval
> short.  This is the kind of problem that will always exist with transparent
> IMs.
> This would be part of the initial system tuning process -- implementation
> detail.
>
> MWS:  There was a suggestion a few days ago that eliminates the question of
> retry
> by the IM's.  I will post something when I catch up with this blizzard of
> postings.
>
>      retryInterval(end) > sum(retryInterval(IM)*retries)
>
> I am not saying DR is an RM artifact.  You objected previously so I am no
> longer
> saying DR is part of RM.  I changed the automatic retry to "decides*" to
> indicate that this happens by some unspecified means, not RM (read note at
> bottom) and not necessarily by retryInterval -- implementation decision.
> </df>
>
> > using ID 1.1.  B will not see this as a duplicate and will pass it on.  B
> will
> > also retry to C using ID 1.0.  C will see ID 1.0 as a duplicate so it
> will not
> > send on but C will send an Ack back to B (who gets it).  C will also
> receive
> ID
> > 1.1 and send it on since it is not a duplicate.  Meanwhile D has finally
> made
> > the send to E and the DR and Ack have come back from E.  D then receives
> ID
> 1.1
> > from C and sends it on since it is not a duplicate.  E receives ID 1.1
> but
> since
> > E is the end, it sees ID 1.0 and ID 1.1 as duplicates (it only uses
> MessageId).
> > E sends an Ack to D for ID 1.1.  E sees ID 1.1 as a duplicate so it does
> not
> > pass on to its application but it does send another DR back to A.  A
> finally
> > receives two DRs, one for ID 1.0 and another for ID 1.1.
>
> So now A has to wait for the total possible number of DRs from E before it
> can
> stop worrying about the message. Why is A allowed to retry before it has
> any indication that there is a problem? The same retryInterval as might be
> used between A and B doesn't seem realistic to be reused between A and E
> as there may be a wide discrepancy between the expected response from B
> and the expected response from E. E may be only intermittently connected
> for any variety of reasons.
>
> <df>A will stop worrying about the message when it gets a DR (if
> requested).
> Why someone would retry is an implementation/user decision and probably not
> automated.  We are leaving this decision criteria unspecified.
> </df>
>
> Thus, a separate parameter is needed to specify how long to wait for
> a DR (which is still optional as far as I'm concerned) or possibly the
> expected business response. As I understand, these are attributes that
> are presently expressed in the BPSS (timeToAcknowledgeReceipt
> and timeToPerform).
>
> MWS:  I have previously mentioned unless the value of the  timeout the
> MSH uses to decide that an acknowledgment isn't coming is made known (CPA?)
> and
> the start of the RetryInterval time is defined relative to other events,
> calculations with all these times are extremely iffy.  I suspect that the
> timeout I am talking about is for receipt of an ACK, not a DR (which I
> believe is in the business level and in any case, optional).
>
> <df>Chris, we are leaving DR as optional and not part of RM -- at your
> request.
> There probably does not need to be a separate parameter because this may
> not
> be/probably won't be automated -- again at your request.   Would you prefer
> an
> automated solution?
> </df>
>
> >
> > *The retry on lack of DR may be either manual or automatic --
> implementation
> > dependant.
> >
> >
> ----------------------------------------------------------------------------
>
> ------------------------
> >               Name: RM9.jpg
> >    RM9.jpg    Type: JPEG Image (image/jpeg)
> >           Encoding: BASE64
>
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>
>
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>