OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

wsrm message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: Re: [wsrm] group management, and dup elimination

Message ExpiryTime is under control of the sender.  If  the receiver 
keeps an ongoing calculated
statistic of the "maximum expiry time over all messages received in this 
group", and uses that as the
GroupExpiryTime, it will be a more conservative algorithm than any of 
the three in Jacques proposal.

The MaxIdleInterval was there to terminate open ended sequences.  
Keeping a current statistic on the maxExpiryTime for messages received 
for this group also terminates open ended sequences.

This also takes away what would be a source of error, users not knowing 
how to determine
the correct groupExpiryTime, before starting a group.

Tom Rutt wrote:

> Jacques Durand wrote:
>> Here is a proposal which, I believe, should address all concerns 
>> heard this afternoon,
>> first and foremost from Sunil. (Close to what he proposed in fact, 
>> but with nicer handling of "corner cases" :)
>> It looks a bit detailed, but I believe is in fact very "implementable".
>> Jacques
> This proposal is very good, since it solves the problems we had.
> Just for thought, what if we took "group expiry time" and 
> GroupMaxIdleTime out of the
> picture, and just keep the ongoing "MaxExpiryTimeInGroup" statistic as 
> the trigger for discard.
> I do believe the behaviour of this approach is the same, and it allows 
> all groups to be "open ended"
> yet "under control".
> Tom Rutt
>> ----------------------------------------------------------------------------------------------- 
>> Background:
>> Let us start with precise semantics: We had 3 possible (non -exclusive)
>> semantics for expiryTime:
>> (1) business semantics (how long a message still makes sense for the 
>> application)
>> (2) RMP management semantics (how long should we try to make 
>> available to the app
>> before giving-up)
>> (3) dup elim semantics: when "at most once" is required, a message 
>> should be subject
>> to dup elim as long as it has not expired.
>> So far, our spec mostly talks of (2).
>> Several of us believe it is none of RMP business to enforce (1).
>> As for (3), there is an obvious impact from (2), as by definition, a 
>> duplicate of
>> an expired message would never be delivered, even if we skip the dup 
>> check.
>> That mostly spares us a dup check.
>> General approach: we endorse semantics (2) and (3).
>> For individual messages (group singletons), dup check is done over 
>> all past
>> messages that have not expired yet, and we keep this mesg ID store 
>> up-to-date.
>> But for groups, we do not want to have to store each message expiry 
>> time,
>> along with message IDs (GroupId + seq number). That would undermine the
>> fast and space efficient check that groups are supposed to provide in 
>> general.
>> General Proposal:
>> - we keep everything of what has been "decided" (or formulated for 
>> decision) so far.
>> - we still assume each individual message of a group has expiryTime.
>> Like for singletons, a message in a group would not be delivered to 
>> app if it has expired
>> when received. However we do NOT memorize this expirtyTime after 
>> first check.
>> (of course, in case we need to keep all undelivered messages of a 
>> broken sequence on receiver,
>> we keep the entire messages and therefore their expiryTime. We decide 
>> to give-up
>> if the min of all expirtyTimes of waiting messages has expired.)
>> - for the sake of simplicity, the receiver RMP will keep the ID of 
>> all past
>> messages for an active group, whether expired or not.
>> That is, once they have been delivered to app, RMP will NOT try to
>> manage expiry time of past delivered messages, and will NOT discard 
>> from memory
>> the message IDs of those expired, like it would for singletons.
>> Duplicate checks would be made over ALL past messages for the group, 
>> whether expired or not.
>> There is no added cost in fact, as these IDs are remembered as seq 
>> num intervals.
>> Will even be faster, as not discarding means fewer intervals.
>> Refined Proposal when groups terminate:
>> NOTE: In the 3 following termination cases, "discarding" a group 
>> means removing
>> everything (state, msg IDs, message bodies), and not doing anymore
>> duplicate checks for these. Any late message from this group will be 
>> treated
>> as from a new group.
>> Termination (t1): when a group terminates because its GroupExpiryTime 
>> is over,
>> it is completely discarded from RMP (state, message Ids and payloads).
>> No duplicate check is done against that group ever. If a "late 
>> duplicate" arrives,
>> it would be automatically discarded due to its expiry time.
>> Termination (t2): when a group terminates because its MaxIdleTime is 
>> past,
>> its messages may not be expired yet. We do not keep expiryTime of 
>> each message
>> of the group, however we keep the max of all expiryTimes of messages 
>> received
>> so far. We use this max as the GroupExpiryTime, and wait for this 
>> time -if needed-
>> before discarding the group. We fall back then to termination (t1).
>> In case a new message arrives after maxIdleTime expires but before 
>> the group expires,
>> we need to define what to do: I suggest we process it and keep 
>> bumping up the max
>> of all expiryTimes if needed (so dup check still enabled for the group).
>> Termination (t3): when a group terminates because it receives a 
>> status="end" message.
>> Note that this only tells us "no greater seq number will be received 
>> after".
>> Like in (t2), its messages may not be expired yet. We treat it like 
>> in (t2): the max
>> of all expiryTimes tells us when the group should be discarded, and 
>> not subject
>> to duplicate check. The group may still have missing messages that 
>> are late. Subcases:
>> (3.1)If the group was complete, still use max of expiryTimes before 
>> discarding it.
>> (so dup checks still possible, but we know are useless after that.)
>> (3.2) If group has missing messages, wait for these to the extent 
>> allowed by
>> either GroupExpiryTime, or maxIdleTime  (whichever was specified), 
>> then fall back on
>> either (t1) or (t2) respectively.

Tom Rutt		email: tom@coastin.com; trutt@fsw.fujitsu.com
Tel: +1 732 801 5744          Fax: +1 732 774 5133

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]