[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [ws-rx] NEW ISSUE: Remove LastMessage
Lei Jin wrote: > I think Anish is talking about releasing the actual store, not the > individual messages inside of it. This concern seems too implementation > specific to me. > Right. I'm not sure that this is too implementation specific. It is certainly possible to build implementations that won't care about such an optimization. But this really depends on the operating environment. It is possible that we (or I) are really looking at premature optimization (which per Knuth is the root of all evil) here. But I'm not convinced (yet) that this is indeed so. Given that TerminateSequence is unreliable (for which Bob/Tom has already raised an issue -- which if fixed makes my concern marginal) and that the expiration time can be PTOS, this may indeed be a problem. This is sort of similar to the outcome of the InactivityTimeout discussion, where we decided that the inactivity timeout did not make sense but a timeout for the 1st message to arrive after CS/CSR exchange did. This was because the CSR message may be lost resulting in an orphaned sequence. Cheers! -Anish -- > Lei > > -----Original Message----- > *From:* Doug Davis [mailto:dug@us.ibm.com] > *Sent:* Wednesday, November 30, 2005 4:09 PM > *To:* ws-rx@lists.oasis-open.org > *Subject:* Re: [ws-rx] NEW ISSUE: Remove LastMessage > > > Anish, > more inline... > thanks > -Doug > > > Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote on 11/30/2005 > 06:16:53 PM: > > > Ok, perhaps I'm not being clear about my concern. Apologies if I'm > > completely missing your point. > > > > 1) I'm *not* worried about the security issue. I agree with you that > > using the LM marker to prevent hijacking of the sequence is an > > inadequate solution. People should not rely on that. > > > > 2) I'm *not* saying that if the RMD receives a message with a LM > marker > > and it has received all the messages in the sequence that it can > release > > *all* its resources. It still has to hang around to ack messages > and can > > only terminate the sequence when it receives the TerminateSequence > > message. Without that (TerminateSequence) the RMD cannot make any > > determination about which acks have been received by the RMS. > > > > 3) What I'm saying is the following - > > When: > > a) an LM marker is present in the last message, and > > b) the RMD receives all the messages upto and including the > message with > > the LM marker. I.e. all the messages in the range [1, > Seqno(LM-message)] > > then: > > at that point the RMD knows that it has received all the messages > in the > > sequence. In certain implementations it can make the > determination that > > it no longer needs the message store (it still needs to be around > to ack > > messages) -- assuming that the messages have been successfully > delivered > > to the AD. i.e., it can give up its message store resource at > this point > > but *not* terminate the Sequence. > > Why would it need to keep them around w/o the LM marker? Once a > message > has been delivered to the AD why would the RMD need to keep it > around at > all - regardless of the LM marker? > > > Without the LM marker, it will never know if it has received all the > > messages in the Sequence until it receives the TerminateSequence > message > > (which is unreliable). > > Under what situations would the RMD not deliver a message to the AD > when it > has all of the messages from 1 to X and then NOT clear up its store > for those > messages? Once it has 1 it can deliver it and erase it, once it did > that > it can do it for 2, then 3.... marking X as the LM doesn't change that. > > > To take the previous example, there are 3 messages in the sequence - > > > > scenario A: LM marker present in message # 3, RMD has received > all the > > messages (1, 2, 3) but not the TerminateSequence. RMD knows that > there > > will never ever be message number 4 (and beyond) and therefore > does not > > need the message store (but still needs to hang around to ack). > > Doesn't need it for 1,2 and 3 at all anyway since there's nothing > stopping > it from deliverying all 3 to the AD and erasing it from its store. > > > scenario B: LM marker is not present in message # 3, RMD has > received > > all the messages (1, 2, 3) but not the TerminateSequence. RMD > does not > > know that there won't be a message number 4 (and beyond) and > therefore > > may indeed need the message store (say if 5 were to arrive before > 4 and > > the DA was inOrder, it will have to store 5 till it receives 4 > and then > > delivery both). > > But it can still deliver and erase 1, 2 and 3 - same as above. LM > doesn't > change that. > > > So I think that the LM marker does allow the RMD to free up certain > > (though not all) resources when it has received all the messages > in the > > sequence and before the TerminateSequence message is received > (this of > > course would depend on the implementation, YMMV). > > > > Hope that makes sense. > > > > Thx. > > > > -Anish > > -- > > > > Doug Davis wrote: > > > > > > Anish, > > > more inline - I think we're getting closer :-) > > > -Doug > > > > > > > > > Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote on 11/29/2005 > > > 10:43:05 PM: > > > > > > > Hey Doug, > > > > > > > > More comments inlined below. > > > > > > > > I realize that I'm picking on an optimization feature which is > > > > applicable only a certain scenario. But I want to explore > this to ensure > > > > that we are indeed doing the right thing. One of the reasons I'm > > > > concerned about this optimization feature is because a > Sequence may not > > > > have an expiration time (PTOS), in which case the loss of > the unreliable > > > > SequenceTermination message may result in the RMD holding on > to the > > > > message queue much longer than needed. More below. > > > > Thx! > > > > > > > > -Anish > > > > -- > > > > > > > > Doug Davis wrote: > > > > > > > > > > Anish, > > > > > comment inline. > > > > > thanks > > > > > -Doug > > > > > > > > > > Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote on > 11/28/2005 > > > > > 09:48:10 PM: > > > > > > > > > > > I thought this issue arose because of the addition of > the Close > > > message > > > > > > and I'm not sure why the Close message changes anything > wrt > > > LastMessage > > > > > > marker. > > > > > > > > > > I actually agree. I don't think Close changes the need > for (or > > > lack of > > > > > a need > > > > > for) LastMessage but if it help convince others then ok ;-) > > > > > > > > > > > > I think your argument actually adds to the reasons > why it > > > should be > > > > > > > removed :-) > > > > > > > You're placing some additional semantics on the > LastMessage > > > marker > > > > > that > > > > > > > are not in the > > > > > > > spec and removing this marker would remove the > possibility of > > > others > > > > > > > reading too much > > > > > > > into it too. > > > > > > You are right that the spec does not explicitly say > that but it > > > seems > > > > > > like a corollary to what the spec says. > > > > > > > > > > Not to me - and this ambiguity is part of the reason to > remove it. > > > > > > > > > > > > > Not sure what additional semantics I'm placing on > LastMessage marker. > > > > If I get a message with LastMessage marker, I know that this > is the last > > > > message in the sequence => RMD cannot get a message with > SeqNo larger > > > > than the SeqNo of the message containing LastMessage marker. > > > > > > Yes - as I noted in the original issue text this would just > prevent the > > > RMD from receiving message with a higher #, but that's about > it. So, > > > if security is the reason for this feature, then its a poor way > of adding > > > security and the other means (SC/Trust) are far more appropriate. > > > > > > > This allows > > > > the RMD to shed some resources, if (and only if) it has > received all the > > > > messages between SeqNo 1 and the SeqNo of the LastMessage. > > > > > > > > Perhaps you are referring to my email where I say: > > > > "... In this case, the loss of the TerminateSequence message > does not > > > > result in an abnormal termination of the sequence on the RMD > side on > > > > time out and no corrective action is necessary on the RMD side." > > > > > > > > You are right, that this is an incorrect conclusion to draw. > This would > > > > still be an abnormal termination. But that doesn't > necessarily take away > > > > the optimization potential at the RMD (assuming all the > messages upto > > > > the LastMessage are received). > > > > > > I'm still not following you. What resources do you think the > RMD can > > > shed based on LastMessage? I think the situation you're > focusing on is > > > one where all messages have been received by the RMD. So, in > that case > > > all you need to keep track of is the highest message number > received, and > > > note that you don't really need to keep around anything else - > just the > > > number itself. Whether or not this one is tagged as the last > message > > > doesn't change that nor can the RMD do anything with that > information > > > aside from not allow any message with a higher # (as mentioned > above). > > > But in terms of freeing resources whether or not the RMD can > free any > > > resources is not influenced by this additional flag. > > > > > > > > > > The LastMessage marker has no impact on the > termination of the > > > > > > > sequence. Even > > > > > > > if the RMD gets a LastMessage marker but not a > Terminate, and it > > > > > chooses > > > > > > > to timeout > > > > > > > the sequence, its still an abnormal termination. > > > > > > > > > > > > Quite true, because the RMS may not have received all > the Acks. > > > But the > > > > > > RMD may not bother to do anything about it and leave it > to the > > > RMS to > > > > > > take correctly action, if necessary. Why is it that > only the RMS can > > > > > > initiate corrective action (as you state below)? > Wouldn't that > > > depend on > > > > > > the application/context. > > > > > > > > > > Any corrective action an RMD can take (which I think is > limited to > > > > > closing or terminating a sequence) is not based on the > LastMessage > > > > > tag/marker. > > > > > > > > > > > > > Why is that? > > > > If the RMD receives all the messages in a Sequence (and it > knows that > > > > from the LM marker), and there is a abnormal termination > (cause the RMD > > > > did not receive the TerminateSequence message), RMD may > decide not to > > > > worry about any corrective action. OTOH, the RMD may decide > to take a > > > > corrective action if it knows that it hasn't received all > the messages > > > > in the Sequence (and LM is the key in figuring that out). > > > > > > Ok, so if a sequence has 3 messages there are two situations, > one where > > > the LM was sent and one where it was not. And then, in both > cases, there > > > is an abnormal termination. I claim that the behavior of the > RMS and RMD > > > in both cases would be the same. The RMS, whether or not the > LM was used, > > > with either be ok with the ACKs it had received or not. On the > RMD side > > > I think you're trying to say that the RMD can think of it as > being "less > > > serious" if the highest message was tagged with the LM marker > and therefore > > > could just drop the sequence, right? > > > I don't think you can make this assumption though. Since the > RMD has not > > > received a terminate the only safe thing it can assume is that > the RMS > > > has not received all of the ACKs - so while I think what you're > saying is > > > that the RMD can drop the seq since it got the LM marker, it > can't since > > > it can't make any assuption about what the RMS knows. And, > therefore would > > > not be able to drop any more knowledge of the sequence since it > must still > > > respond to Close/Terminate messages. > > > > > > > > > > The only non-abnormal > > > > > > > termination > > > > > > > is when the RMD receives a Terminate from the RMS. > And,either > > > way, > > > > > the > > > > > > > RMD needs > > > > > > > to keep the sequence state around until it gets a > Terminate or it > > > > > > > decides to terminate it > > > > > > > on its own. Receipt of a LastMessage does not change > this. > > > > > > > > > > > > But the RMD can relinquish the message store/queue at this > > > point. All it > > > > > > has to remember is the Seq Number for the LastMessage. > > > > > > > > > > What the RMD needs to remember isn't dependent on > LastMessage. > > > Let's say > > > > > the RMS sends messages 1, 2 and 3 but 3 is lost. The RMD > will > > > remember > > > > > that > > > > > 1 and 2 arrived. Upon receipt of an AckReq > (w/LastMessage=3) what will > > > > > it do > > > > > with this knowledge? Nothing. It will still send back an > Ack with > > > message > > > > > numbers 1 and 2. Knowing that the RMS sent any message # > higher > > > than 2 > > > > > doesn't > > > > > change the state information of what the RMD needs to keep > around. > > > How > > > > > much > > > > > of each message the RMD needs to remember (just state vs > entire > > > message > > > > > content) > > > > > is more tied to the InOrder flag - so it needs to remember > the entire > > > > > message > > > > > until the time it can actually deliver the message to the > RMD's app. > > > > > Again, > > > > > LastMessage has no impact on this. > > > > > > > > > > > > > I didn't quite state the situation correctly: > > > > The RMD can relinquish the message store/queue only when it > knows it has > > > > received all the messages. This requires that there are no > 'holes' *and* > > > > the knowledge of the seqno of the last message. > > > > I.e., if a message within the sequence was not received > (received > > > > messages: 1, 2, 3, 5, 6) then knowing that 6 was the last > message does > > > > not provide any ability for the RMD to optimize. > > > > > > See above - previous comment. > > > > > > > > > > Ultimately, > > > > > > > its up to the > > > > > > > RMS to decide whether or not a sequence needs to have > some > > > kind of > > > > > error > > > > > > > recovery > > > > > > > done, not the RMD, and this would be based on the > Acks it > > > receives and > > > > > > > not the delivery > > > > > > > of a Terminate (or LastMessage marker) to the RMD. > > > > > > > > > > > > > > As to freeing up some resources, the LastMessage > doesn't > > > change this > > > > > > > either. The data > > > > > > > an RMD retains isn't dependent on the LastMessage. > You seem to > > > > > indicate > > > > > > > that the RMD > > > > > > > can free up some stuff base on this marker - this > isn't true. > > > > > > > > > > > > Any reason not to relinquish the message store, close > > > > > > connections/sessions, commit transactions etc? > > > > > > > > > > Perhaps but its not based on LastMessage but rather other > information, > > > > > like which messages have arrived and been delivered (as > mentioned > > > above). > > > > > > > > > > > > > So if the RMD has received all the messages (including the > last message > > > > in the Sequence) and if the LM marker is not present the RMD > cannot > > > > relinquish the message store/connection/session etc unless > either it > > > > timesout or expires (and not all Sequences have expiration > time) or the > > > > TerminateSequence message (which is unreliable) is received. > > > > > > See above. > > > > > > > > > I'm assuming that the motivation for getting rid of > LastMessage > > > marker > > > > > > is so that the RMS is not forced to send an additional > empty message > > > > > > when it is done. For some applications, it is not clear > which > > > the last > > > > > > message with payload is, until it has finished sending > that message. > > > > > > > > > > I don't think it matters. If after message #3 is sent, > the RMS decides > > > > > that that really was the last message then it can send a > Terminate > > > once it > > > > > receives the Ack for all messages (1-3). Having to send > an additional > > > > > message > > > > > to say 'tag message 3 as the last one' doesn't change any > message > > > > > processing > > > > > logic on either side. In either case the RMD will still > need to > > > > > maintain the > > > > > exact same state information. I'm claiming that any > information > > > that you > > > > > think the RMD can relinquish upon receipt of the > LastMessage could have > > > > > been dropped before the receipt of the LastMessage marker. > > > > > > > > > > > > > The RMD cannot drop the message store without the LM marker > (assuming > > > > that there are no 'holes') unless it receives a > TerminateSequence > > > > message or timesout/expires. > > > > > > See above - LM marker doesn't change anything. > > > > > > > > > What if we were to make the LastMessage marker > optional? Allow > > > RMS to > > > > > > include it and the RMD can optimize things based on the > marker if it > > > > > > wishes to do so, but not require the RMS to always send it. > > > > > > > > > > I still don't see what optimizations you think the RMD can > do with it? > > > > > I don't think there are any - which is why I think its a > useless > > > feature.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]