Rel 50, 52, 57 revised

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Title: Rel 50, 52, 57 revised

Here is the revised version of Rel50, 52, 57 if ExpiryTime is "latest time at which a message can be delivered to application".

(I tagged them "solution b", to distinguish them from previous proposal.)

The main changes are for Rel50 of course, and also Rel57 which now can rely on ExpiryTime for aborting an out-of-order
sequence.

I think the crux of this alternative, is really what do we want ExpiryTime to mean.

There are actually three alternatives, compared with the two (1) and (2) I suggested yesterday,
and it could very well be that the 3rd one is the one that is most appropriate for RM, and
that we can all agree with:

(1) pure transport QoS semantics: time at which a message is expected to have arrived to RMP.
See "previous" solution of Rel50, 52, 57.

(2) application semantics: time at which a message is expected to be useful to application,
based on its business value.
(I really believe that is out of scope of this spec) (solution "b" applies)

(3) Delivery QoS (includes transport time and other RMP processing, like Ordering):
time at which a message is expected to be delivered to application,
based on expected performance of transport + RMP.
solution "b" of Rel50, 52, 57 here attached, applies.

Regards,

jacques
<<Time-Rel50b.txt>> <<Time-Rel57b.txt>> <<Time-Rel52b.txt>>

Rel 50: (solution b) Semantics of ExpiryTime:
-------------------------------

Proposal:

ExpiryTime indicates the ultimate date after which the receiver RMP MUST
not make a received message available to the application. 

After a message has been sent for the first time, ExpiryTime in a message 
MUST NOT be modified in any case by the Sender, when resending the message:
two messages with same ID (duplicates) MUST also have same ExpiryTime.

When a message expires on the Sender side before being sucessfully sent,
a Sender RMP MUST NOT send it or resend it, and must communicate a delivery failure 
to the Sender application.

When a message received by an RMP expires before being delivered to the
application, and if guaranteed delivery is required, an RMP MUST make available
to the Sender a fault message, with Fault code "MessageExpired".

When receiving a message that has already expired, an RMP MAY decide to
not accept the message. In that case, and if guaranteed delivery is required, 
the RMP will not generate a positive Acknowledgement and will directly generate 
a fault instead.

NOTES: 
- It is possible that a positive acknowledgement has been generated for a message 
that is later discarded by the RMP, e.g. due to expiration. In such case a Fault is
always generated to the Sender.
- Given the above definition of ExpiryTime, in case duplicate elimination is required, 
when a received message is processed, it is sufficient to only check for its duplicates 
among IDs of past messages that have not expired yet at the time of the duplicate check. 




Rel 57: (Solution b) Ordering and Missing Message Behavior
---------------------------------------

Proposal:

[short answer]: 
the RMP will wait until expiration of the earliest ExpiryTime of the 
out-of-order messages. All out-of-order messages will then be discarded.
However, a shortage in storage resources may lead to discard out-of-order 
messages before the group terminates. 
In that case, a resource fault (Fault code: "MessageStoreOverflow" will be 
generated and made available to the Sender. 

[long answer]:
Given the definition of Guaranteed Ordered Delivery, a message must not be delivered
to the receiver application if not all previous messages were.
Therefore in case a Receiver RMP decides to not wait any longer for a missing
message, the already received out-of-order messages for the group MUST not be delivered 
to the application, and can be discarded. 

On the Receiver side, as soon as the earliest ExpiryTime of pending (stored) 
out-of-order messages has expired, all the present and future out-of-order messages 
for this group MUST not be delivered to the application. 
The RMP MUST terminate the out-of-order group as early as convenient after this date, 
and out-of-order messages must be discarded. 
(see Rel 52 for definition of group temination)

On the Sender side, the RMP MUST terminate an ordered group as early as possible after
one of these events occurs, whichever comes first:
(1) a message for the group has failed to be delivered. 
(no Acknowledgement has been received yet after the last retry time plus retryinterval)
(2)the ExpiryTime of a message to be sent for the group, has expired before the message 
is sent or before it has been acknowledged.
Deciding that a message has "Not been acknowledged" must require a failed poll request 
from the Sender, in case the reply pattern is "poll", or must exhaust the expected
delay in receiving a multiple-acknowledgement.

NOTE: discarding of out-of-order messages of a group, and more generally terminating a 
group, does not mean yet the removal of the group state, as message IDs may still 
be needed for duplicate checks. 


Rel 52: (Solution b)(New rules for persistence, also, related "timing issues") 
-------

Proposal:

Q1: When can a recipient "forget" about a sequence? 
How long must an out-of-order message be stored prior to giving up on 
ever delivering it to the application?

Answer: see Rel57.

Q2: How do these time windows relate to the TTL (expiration) of an individual message?

Answer: see Rel50 and Rel57.

Q3: Within what window will a duplicate be detected? 

[Short Answer:] duplicates will be detected over all non-expired messages. 
Since by definition, messages that are expired before the RMP processing is over 
will not be made available to the application, and because ExpiryTime is identical across
duplicates, there is no risk ever to deliver two duplicates to the receiver application, 
when restricting the duplicate check over the IDs of non-expired messages. 

[Long answer]: 

General rule for persisting the ID of a message:
---------------------------------------------

The duplicate elimination feature requires persisting the message ID
(GroupId and optionally the Sequence Number), after the message
has been processed and delivered (made available) to the application.
The general rule about message ID persistence, is that an RMP
MUST persist the ID of a message at least until the message expires.

It is RECOMMENDED that an RMP removes the ID of a singleton message 
(message with GroupId but no SequenceNumber) from its persistent store after it expires.
However, unlike for singletons, it is RECOMMENDED to NOT remove the ID of a message that
belongs to a non-singleton group, when the message expires. The reason is, message IDs
for groups can be stored efficiently as intervals of integers.

NOTE: For groups, duplicate checks may be made over all past messages of the group, 
whether expired or not, without loss of performance. The check will in fact be faster, 
as not discarding expired IDs means fewer intervals to look up.

General rules for terminating a group, and for removing the state of a group:
-----------------------------------------------------------------

NOTE: these termination rules apply to both ordered and unordered groups.

NOTE: In all termination cases (t1, t2, t3, t4) below, it is not necessary to remember 
the ExpiryTime of all messages of a group, as only the maximum of all ExpiryTime values 
is needed.

NOTE:Termination of a group, and removal of its state are two distinct events.
- When a group terminates, both in sender and receiver, no message is expected to be
sent or received for this group anymore. If a message is received after the
termination of a group, with same GroupId as the terminated group, it is considered
by the Receiver as belonging to a new group (the Receiver is not required
to verify that a new GroupId value has not been used in a previous group, as this test 
is impractical).
- When the state of a group is removed, any trace of the group (including status, groupId, 
current sequence number, as well as all received message IDs for the group 
e.g. SequenceNumber intervals) are removed, and therefore not available anymore for 
duplicate checks. State removal may occur either at termination time or later. 

Termination (t1): 
Triggering event: GroupExpiryTime is over.
Receiver side: The RMP MUST NOT accept any new message for this group and MUST
terminate the group. It is RECOMMENDED that its state be removed as soon as possible.
No duplicate check needs to be done against that group ever. If a "late duplicate" arrives,
it would never be delivered to the application, as its ExpiryTime, which is always earlier 
than GroupExpiryTime, would have expired.
Sender side: The group MUST be terminated, and its state removed from the RMP.

Termination (t2): 
Triggering event: GroupMaxIdleTime is over.
Receiver side: Except for an ordered group that is out-of-order (see this case in Rel52), 
the group MUST be terminated. But unlike (t1), some of its 
past messages may not have expired yet, and therefore their ID still be needed 
for duplicate checks. If we define max(all ExpiryTime) as the max of all ExpiryTimes 
of messages already received for a group, an RMP MUST persist the state of a group
even after termination of the group, at least until max(all ExpiryTime), in case 
duplicate elimination is required.
Sender side: The group MUST be immediately terminated, and its state is removed from the RMP
when either (1) the time elapsed since the last sent message (including retries) exceeds 
GroupMaxIdleTime, or (2) for an ordered group, a message has failed to be successfully 
delivered (as described in Rel 57), whichever occurs first.

Termination (t3): 
Triggering event: the RMP receives a status="end" message. The group had either
GroupExpiryTime or GroupMaxIdleTime specified.
Note that such an event only tells that "no greater seq number will ever be received after", 
but late messages may still arrive for this group. 
Subcase t3.1: The group was complete on receiver side. 
Receiver side: The group MUST be terminated. 
However, its state is removed according to (t1) or (t2), depending which termination
criterion was given.
Sender side: The group is also known to be complete, and MUST be terminated. 
However, if guaranteed delivery was required, all sent messages must either have been
acknowledged, or been faulted, before termination. 
Subcase t3.2: The group was not complete on receiver side. 
Then the receiver RMP and sender RMP MUST apply rules of (t1) or (t2), depending which one 
of GroupExpiryTime or GroupMaxIdleTime was specified.

Termination (t4): 
Triggering event: the RMP receives a status="end" message. The group had neither
GroupExpiryTime nor GroupMaxIdleTime specified.
Subcase t4.1: The group was complete on receiver side. 
Receiver side: The group MUST be terminated. The date of removal of its state 
is based on rules for (t2).
Sender side: see subcase t3.1 above.
Subcase t4.2: The group was not complete on receiver side.
This event, by itself, does not cause the termination of the group.