RE: [wsrm] Need to rethink semantics of duplicate elimination behaviour

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Title: RE: [wsrm] Need to rethink semantics of duplicate elimination behaviour

inline:

-----Original Message-----
From: Doug Bunting [mailto:Doug.Bunting@sun.com]
Sent: Tuesday, December 16, 2003 5:02 PM
To: wsrm
Subject: Re: [wsrm] Need to rethink semantics of duplicate elimination
behaviour

Jacques,

I have not reviewed the most recent text at this level of detail but
find it likely some portions have not yet been updated to reflect the
new "ack-on-delivery" semantics.  I agree the RMP should not send an
acknowledgment for any message until its delivery.  (And, I hope
"delivery" is consistently used and well defined in the specification.)

I am not sure I follow you after "Personally ..." since duplicates may
certainly arrive both before and after delivery.  A duplicate received
before delivery is completely ignored, closing the inbound connection
or, possibly, holding it open until the acknowledgment is available.

<JD> I think we agree, "completely ignored" means neither acked nor faulted
(i.e. just dropped).
If you mean "HTTP connection", from HTTP viewpoint we may still have
to work out details, but these are - I believe - orthogonal to our main
line of RM logic here. For example, even for a discarded duplicate,
an HTTP response (e.g. empty) should be sent, to properly close the transaction.
The initial message could be the one holding up the connection if not delivered yet
in cases of sync reply for Ack sending (e.g. "response" replypattern.)
 Not to get in too much detail, there is also the possibility of pipelining requests in HTTP1.1 which would require sending a batch of responses in same

order as HTTP requests were received. So a discarded duplicate may still leave
a trace at HTTP level, for a batch of delayed responses to be sent later
(and that batch may have to wait an out-of-order sequence to get back in order
and delivered before being sent).
</JD>
 

 A duplicate received after delivery is acknowledged, identically (from the
sender's perspective) to handling of the same message when nothing
blocks immediate delivery.  Are we agreeing or not?

<JD> agree with this (though not apparent in current spec, and
yet to be approved by TC?) but with some reserve:
(1) I see here an optimization only, not really a critical feature,
(2) after thoughts, not sure whether we should make that a MUST vs. a
 RECOMMENDATION only: that would require to maintain new status for each
 stored message ID, whether it was delivered vs. non-delivered,
so we know when to send an Ack or not if a duplicate comes in.
In case of groups, we have managed to avoid storing any
individual message data (big benefit  e.g. for memory-constrained devices)
e.g. IDs are stored as integer intervals, and we need only one "expiration" value
max(ExpiryTime). So for groups, if the only case of non-delivered-ID-in-store
is out-of-order sequences, we can still make this test without a new
boolean for each message. But is that the only case of not-yet-delivered
mesg with ID in store? I know all this sounds very implementation-centered,
but in the case of a non-critical feature, I would hate to break an
important implementation benefit of groups...
</JD>

The expiration question is interesting but somewhat orthogonal.  If a
message is received, processed appropriately and acknowledged at some
time and the same message (a duplicate) is received again after it has
expired, what should happen? 

<JD> the current logic (which I find actually brilliant !) takes care of
that I believe:
An RMP does not have to keep in memory the IDs of expired messages,
which has two benefits: (1) especially useful to control the ID store size
in case of many singletons, (2) this in turn will keep the dup detection
tractable. Indeed, dup detection will not be possible anymore (in general)
on expired messages, but precisely we don't need it because an expired
message coming in would anyway never be delivered. So dup or not,
we don't need to know: all we care about is that it would never be delivered
to app in both cases...
So answer to your question, as of today, is that we would have to treat
an expired incoming dup like any other expired message... because we have no way
anymore to know its a dup. (so no Ack...).
</JD>

The most correct information the receiver
could supply would be an acknowledgment since the original message was
processed within its expiration window.  The easiest information the
receiver could supply would be an expiration fault since the duplicate
contains enough data to notice that fault without any additional checks.
  I would prefer that system return the acknowledgment if possible --
that is, prior to removing the necessary information from its store.

<JD> due to allowed removal of state/ID after msg expiration, little can be done
for expired dups... but I don't see that as an issue:
we can make it part of the semantics of ExpiryTime, that it also delimits the
time window where the best acknowledgement effort is made.
</JD>

that it after message expiration,
Might be a RECOMMENDATION in the text?

thanx,
        doug

On 16-Dec-03 16:23, Jacques Durand wrote:

> Doug:
>
> I suspect a remaining gray area in the Ack semantics and behavior:
>
> - Given we all agree on Ack-on-delivery (meaning the message has
> actually been "delivered" to the receiver app),
>
> What you point at, is the case where:
> - message was well received and well delivered.
> - the Ack was sent back,
> - the Ack was never received by Sender.
>
> What you expect is:
> - when duplicates for the delivered message are received,
> the RMP will repeat the Ack sending, because the message
> was actually previously delivered, yet the Sender does not seem to know
> this.
>
> So the current spec is still unprecise as what "sending an Ack on
> duplicate" means (3.2.1 line 552), since that seems to be a remnant of
> the old
> "ack-on-receipt" semantics:
> with "ack-on-delivery", we should not send an ack for a duplicate
> of a message that has NOT been delivered for whatever reason.
>
> Personally, I thought this ack of duplicate was obsolete
> with the new ack semantics, yet I see your proposal as an optimization that
> may reduce the number of cases where:
> (1) a message was well delivered
> (2) Sender cannot assume it was delivered.
> Note that when a duplicate has expired when arriving, the ID of the initial
> message may be removed, and the receiver RMP may never know that this is
> a duplicate of a well-delivered message (this duplicate would be discarded
> due to expiration.)
>
> I have no problem with acking duplicates in that case.
> That would improve things, yet only an optimization.
>
> Is that a good summary of where we are?
>
> Jacques
>
>
> -----Original Message-----
> From: Doug Bunting [mailto:Doug.Bunting@sun.com]
> Sent: Tuesday, December 16, 2003 2:40 PM
> To: Jacques Durand
> Cc: tom@coastin.com; Bob Freund; Sunil Kunisetty; wsrm
> Subject: Re: [wsrm] Need to rethink semantics of duplicate elimination
> behaviour
>
>
> Jacques,
>
> I am not sure what you mean.  A message may be resent after the
> receiving application has processed it.  The previous acknowledgment
> could easily have been lost.  In effect, the sender is performing the
> resend in order to retrieve the missing acknowledgment.  Of course, that
> sender does not know or care whether the original message or the
> acknowledgment was lost.
>
> This all means we need acknowledgments for all messages sent reliably
> whether duplicates or not.
>
> thanx,
>         doug
>
> On 16-Dec-03 14:31, Jacques Durand wrote:
>
>  > + 1 on no faults for dups.
>  >
>  > But the end of your mail is ambiguous on one point:
>  >
>  >  >I would propose we do not add these dupOfWhatever faults in the first
>  >  >place and simply return acknowledgements.
>  >
>  > given  the Ack semantics agreed on (ack-on-delivery),
>  > no acks will be sent for dups - which is fine.
>  >
>  > Jacques
>
> ...
>

To unsubscribe from this mailing list (and be removed from the roster of the OASIS TC), go to http://www.oasis-open.org/apps/org/workgroup/wsrm/members/leave_workgroup.php.