Re: [ws-rx] NEW ISSUE: Remove LastMessage

[Anish Karmarkar <Anish.Karmarkar@oracle.com>]

Hey Doug,

More comments inlined below.

I realize that I'm picking on an optimization feature which is 
applicable only a certain scenario. But I want to explore this to ensure 
that we are indeed doing the right thing. One of the reasons I'm 
concerned about this optimization feature is because a Sequence may not 
have an expiration time (PTOS), in which case the loss of the unreliable 
SequenceTermination message may result in the RMD holding on to the 
message queue much longer than needed. More below.

Thx!

-Anish
--

Doug Davis wrote:
> 
> Anish,
>   comment inline.
> thanks
> -Doug
> 
> Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote on 11/28/2005 
> 09:48:10 PM:
> 
>  > I thought this issue arose because of the addition of the Close message
>  > and I'm not sure why the Close message changes anything wrt LastMessage
>  > marker.
> 
> I actually agree.  I don't think Close changes the need for (or lack of 
> a need
> for) LastMessage but if it help convince others then ok  ;-)
> 
>  > >   I think your argument actually adds to the reasons why it should be
>  > > removed  :-)
>  > > You're placing some additional semantics on the LastMessage marker 
> that
>  > > are not in the
>  > > spec and removing this marker would remove the possibility of others
>  > > reading too much
>  > > into it too.
>  > You are right that the spec does not explicitly say that but it seems
>  > like a corollary to what the spec says.
> 
> Not to me - and this ambiguity is part of the reason to remove it.
> 

Not sure what additional semantics I'm placing on LastMessage marker.
If I get a message with LastMessage marker, I know that this is the last 
message in the sequence => RMD cannot get a message with SeqNo larger 
than the SeqNo of the message containing LastMessage marker. This allows 
the RMD to shed some resources, if (and only if) it has received all the 
messages between SeqNo 1 and the SeqNo of the LastMessage.

Perhaps you are referring to my email where I say:
"... In this case, the loss of the TerminateSequence message does not 
result in an abnormal termination of the sequence on the RMD side on 
time out and no corrective action is necessary on the RMD side."

You are right, that this is an incorrect conclusion to draw. This would 
still be an abnormal termination. But that doesn't necessarily take away 
the optimization potential at the RMD (assuming all the messages upto 
the LastMessage are received).

>  > >   The LastMessage marker has no impact on the termination of the
>  > > sequence.  Even
>  > > if the RMD gets a LastMessage marker but not a Terminate, and it 
> chooses
>  > > to timeout
>  > > the sequence, its still an abnormal termination.
>  >
>  > Quite true, because the RMS may not have received all the Acks. But the
>  > RMD may not bother to do anything about it and leave it to the RMS to
>  > take correctly action, if necessary. Why is it that only the RMS can
>  > initiate corrective action (as you state below)? Wouldn't that depend on
>  > the application/context.
> 
> Any corrective action an RMD can take (which I think is limited to
> closing or terminating a sequence) is not based on the LastMessage 
> tag/marker.
> 

Why is that?
If the RMD receives all the messages in a Sequence (and it knows that 
from the LM marker), and there is a abnormal termination (cause the RMD 
did not receive the TerminateSequence message), RMD may decide not to 
worry about any corrective action. OTOH, the RMD may decide to take a 
corrective action if it knows that it hasn't received all the messages 
in the Sequence (and LM is the key in figuring that out).

>  > > The only non-abnormal
>  > > termination
>  > > is when the RMD receives a Terminate from the RMS.  And,either way, 
> the
>  > > RMD needs
>  > > to keep the sequence state around until it gets a Terminate or it
>  > > decides to terminate it
>  > > on its own.  Receipt of a LastMessage does not change this.  
>  >
>  > But the RMD can relinquish the message store/queue at this point. All it
>  > has to remember is the Seq Number for the LastMessage.
> 
> What the RMD needs to remember isn't dependent on LastMessage.  Let's say
> the RMS sends messages 1, 2 and 3 but 3 is lost.  The RMD will remember 
> that
> 1 and 2 arrived. Upon receipt of an AckReq (w/LastMessage=3) what will 
> it do
> with this knowledge?  Nothing. It will still send back an Ack with message
> numbers 1 and 2.  Knowing that the RMS sent any message # higher than 2 
> doesn't
> change the state information of what the RMD needs to keep around.  How 
> much
> of each message the RMD needs to remember (just state vs entire message 
> content)
> is more tied to the InOrder flag - so it needs to remember the entire 
> message
> until the time it can actually deliver the message to the RMD's app. 
>  Again,
> LastMessage has no impact on this.
> 

I didn't quite state the situation correctly:
The RMD can relinquish the message store/queue only when it knows it has 
received all the messages. This requires that there are no 'holes' *and* 
the knowledge of the seqno of the last message.
I.e., if a message within the sequence was not received (received 
messages: 1, 2, 3, 5, 6) then knowing that 6 was the last message does 
not provide any ability for the RMD to optimize.

>  > > Ultimately,
>  > > its up to the
>  > > RMS to decide whether or not a sequence needs to have some kind of 
> error
>  > > recovery
>  > > done, not the RMD, and this would be based on the Acks it receives and
>  > > not the delivery
>  > > of a Terminate (or LastMessage marker)  to the RMD.
>  > >
>  > >   As to freeing up some resources, the LastMessage doesn't change this
>  > > either. The data
>  > > an RMD retains isn't dependent on the LastMessage.  You seem to 
> indicate
>  > > that the RMD
>  > > can free up some stuff base on this marker - this isn't true.  
>  >
>  > Any reason not to relinquish the message store, close
>  > connections/sessions, commit transactions etc?
> 
> Perhaps but its not based on LastMessage but rather other information,
> like which messages have arrived and been delivered (as mentioned above).
> 

So if the RMD has received all the messages (including the last message 
in the Sequence) and if the LM marker is not present the RMD cannot 
relinquish the message store/connection/session etc unless either it 
timesout or expires (and not all Sequences have expiration time) or the 
TerminateSequence message (which is unreliable) is received.

>  > I'm assuming that the motivation for getting rid of LastMessage marker
>  > is so that the RMS is not forced to send an additional empty message
>  > when it is done. For some applications, it is not clear which the last
>  > message with payload is, until it has finished sending that message.
> 
> I don't think it matters.  If after message #3 is sent, the RMS decides
> that that really was the last message then it can send a Terminate once it
> receives the Ack for all messages (1-3).  Having to send an additional 
> message
> to say 'tag message 3 as the last one' doesn't change any message 
> processing
> logic on either side.  In either case the RMD will still need to 
> maintain the
> exact same state information.  I'm claiming that any information that you
> think the RMD can relinquish upon receipt of the LastMessage could have
> been dropped before the receipt of the LastMessage marker.
> 

The RMD cannot drop the message store without the LM marker (assuming 
that there are no 'holes') unless it receives a TerminateSequence 
message or timesout/expires.

>  > What if we were to make the LastMessage marker optional? Allow RMS to
>  > include it and the RMD can optimize things based on the marker if it
>  > wishes to do so, but not require the RMS to always send it.
> 
> I still don't see what optimizations you think the RMD can do with it?
> I don't think there are any - which is why I think its a useless feature.