[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [ws-rx] NEW ISSUE: Remove LastMessage
Anish: I still feel this concern is implementation specific. First of all, at least in our implementation, maintaining the store/queue is not costly if we can get rid of all the messages belonging to that particular sequence. While I understand this may not be true in other implementations, I can't imagine this being a significant cost. Secondly, if maintaining a store will cause a significant resource leak, you can always have local implementation strategy to get rid of the store after a period of no message received for that sequence(and there is no gap in the received messages). This presumably will take care of the situation where a TerminateSequence is missing. If you do receive a message later, you can always recreate the store. Lei > -----Original Message----- > From: Anish Karmarkar [mailto:Anish.Karmarkar@oracle.com] > Sent: Wednesday, November 30, 2005 6:59 PM > To: ws-rx@lists.oasis-open.org > Subject: Re: [ws-rx] NEW ISSUE: Remove LastMessage > > > Lei Jin wrote: > > I think Anish is talking about releasing the actual store, not the > > individual messages inside of it. This concern seems too > implementation > > specific to me. > > > > Right. > > I'm not sure that this is too implementation specific. It is > certainly > possible to build implementations that won't care about such an > optimization. But this really depends on the operating environment. > > It is possible that we (or I) are really looking at premature > optimization (which per Knuth is the root of all evil) here. > But I'm not convinced (yet) that this is indeed so. > > Given that TerminateSequence is unreliable (for which Bob/Tom has > already raised an issue -- which if fixed makes my concern > marginal) and > that the expiration time can be PTOS, this may indeed be a > problem. This > is sort of similar to the outcome of the InactivityTimeout > discussion, > where we decided that the inactivity timeout did not make sense but a > timeout for the 1st message to arrive after CS/CSR exchange did. This > was because the CSR message may be lost resulting in an > orphaned sequence. > > Cheers! > > -Anish > -- > > > Lei > > > > -----Original Message----- > > *From:* Doug Davis [mailto:dug@us.ibm.com] > > *Sent:* Wednesday, November 30, 2005 4:09 PM > > *To:* ws-rx@lists.oasis-open.org > > *Subject:* Re: [ws-rx] NEW ISSUE: Remove LastMessage > > > > > > Anish, > > more inline... > > thanks > > -Doug > > > > > > Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote on 11/30/2005 > > 06:16:53 PM: > > > > > Ok, perhaps I'm not being clear about my concern. > Apologies if I'm > > > completely missing your point. > > > > > > 1) I'm *not* worried about the security issue. I > agree with you that > > > using the LM marker to prevent hijacking of the > sequence is an > > > inadequate solution. People should not rely on that. > > > > > > 2) I'm *not* saying that if the RMD receives a > message with a LM > > marker > > > and it has received all the messages in the sequence > that it can > > release > > > *all* its resources. It still has to hang around to > ack messages > > and can > > > only terminate the sequence when it receives the > TerminateSequence > > > message. Without that (TerminateSequence) the RMD > cannot make any > > > determination about which acks have been received by the RMS. > > > > > > 3) What I'm saying is the following - > > > When: > > > a) an LM marker is present in the last message, and > > > b) the RMD receives all the messages upto and including the > > message with > > > the LM marker. I.e. all the messages in the range [1, > > Seqno(LM-message)] > > > then: > > > at that point the RMD knows that it has received all > the messages > > in the > > > sequence. In certain implementations it can make the > > determination that > > > it no longer needs the message store (it still needs > to be around > > to ack > > > messages) -- assuming that the messages have been > successfully > > delivered > > > to the AD. i.e., it can give up its message store resource at > > this point > > > but *not* terminate the Sequence. > > > > Why would it need to keep them around w/o the LM marker? Once a > > message > > has been delivered to the AD why would the RMD need to keep it > > around at > > all - regardless of the LM marker? > > > > > Without the LM marker, it will never know if it has > received all the > > > messages in the Sequence until it receives the > TerminateSequence > > message > > > (which is unreliable). > > > > Under what situations would the RMD not deliver a > message to the AD > > when it > > has all of the messages from 1 to X and then NOT clear > up its store > > for those > > messages? Once it has 1 it can deliver it and erase > it, once it did > > that > > it can do it for 2, then 3.... marking X as the LM > doesn't change that. > > > > > To take the previous example, there are 3 messages > in the sequence - > > > > > > scenario A: LM marker present in message # 3, RMD > has received > > all the > > > messages (1, 2, 3) but not the TerminateSequence. > RMD knows that > > there > > > will never ever be message number 4 (and beyond) and > therefore > > does not > > > need the message store (but still needs to hang > around to ack). > > > > Doesn't need it for 1,2 and 3 at all anyway since > there's nothing > > stopping > > it from deliverying all 3 to the AD and erasing it from > its store. > > > > > scenario B: LM marker is not present in message # 3, RMD has > > received > > > all the messages (1, 2, 3) but not the TerminateSequence. RMD > > does not > > > know that there won't be a message number 4 (and beyond) and > > therefore > > > may indeed need the message store (say if 5 were to > arrive before > > 4 and > > > the DA was inOrder, it will have to store 5 till it > receives 4 > > and then > > > delivery both). > > > > But it can still deliver and erase 1, 2 and 3 - same as > above. LM > > doesn't > > change that. > > > > > So I think that the LM marker does allow the RMD to > free up certain > > > (though not all) resources when it has received all > the messages > > in the > > > sequence and before the TerminateSequence message is received > > (this of > > > course would depend on the implementation, YMMV). > > > > > > Hope that makes sense. > > > > > > Thx. > > > > > > -Anish > > > -- > > > > > > Doug Davis wrote: > > > > > > > > Anish, > > > > more inline - I think we're getting closer :-) > > > > -Doug > > > > > > > > > > > > Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote > on 11/29/2005 > > > > 10:43:05 PM: > > > > > > > > > Hey Doug, > > > > > > > > > > More comments inlined below. > > > > > > > > > > I realize that I'm picking on an optimization > feature which is > > > > > applicable only a certain scenario. But I want > to explore > > this to ensure > > > > > that we are indeed doing the right thing. One > of the reasons I'm > > > > > concerned about this optimization feature is because a > > Sequence may not > > > > > have an expiration time (PTOS), in which case > the loss of > > the unreliable > > > > > SequenceTermination message may result in the > RMD holding on > > to the > > > > > message queue much longer than needed. More below. > > > > > Thx! > > > > > > > > > > -Anish > > > > > -- > > > > > > > > > > Doug Davis wrote: > > > > > > > > > > > > Anish, > > > > > > comment inline. > > > > > > thanks > > > > > > -Doug > > > > > > > > > > > > Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote on > > 11/28/2005 > > > > > > 09:48:10 PM: > > > > > > > > > > > > > I thought this issue arose because of the > addition of > > the Close > > > > message > > > > > > > and I'm not sure why the Close message > changes anything > > wrt > > > > LastMessage > > > > > > > marker. > > > > > > > > > > > > I actually agree. I don't think Close > changes the need > > for (or > > > > lack of > > > > > > a need > > > > > > for) LastMessage but if it help convince > others then ok ;-) > > > > > > > > > > > > > > I think your argument actually adds to > the reasons > > why it > > > > should be > > > > > > > > removed :-) > > > > > > > > You're placing some additional semantics on the > > LastMessage > > > > marker > > > > > > that > > > > > > > > are not in the > > > > > > > > spec and removing this marker would remove the > > possibility of > > > > others > > > > > > > > reading too much > > > > > > > > into it too. > > > > > > > You are right that the spec does not explicitly say > > that but it > > > > seems > > > > > > > like a corollary to what the spec says. > > > > > > > > > > > > Not to me - and this ambiguity is part of the > reason to > > remove it. > > > > > > > > > > > > > > > > Not sure what additional semantics I'm placing on > > LastMessage marker. > > > > > If I get a message with LastMessage marker, I > know that this > > is the last > > > > > message in the sequence => RMD cannot get a message with > > SeqNo larger > > > > > than the SeqNo of the message containing > LastMessage marker. > > > > > > > > Yes - as I noted in the original issue text this would just > > prevent the > > > > RMD from receiving message with a higher #, but > that's about > > it. So, > > > > if security is the reason for this feature, then > its a poor way > > of adding > > > > security and the other means (SC/Trust) are far > more appropriate. > > > > > > > > > This allows > > > > > the RMD to shed some resources, if (and only if) it has > > received all the > > > > > messages between SeqNo 1 and the SeqNo of the > LastMessage. > > > > > > > > > > Perhaps you are referring to my email where I say: > > > > > "... In this case, the loss of the > TerminateSequence message > > does not > > > > > result in an abnormal termination of the > sequence on the RMD > > side on > > > > > time out and no corrective action is necessary > on the RMD side." > > > > > > > > > > You are right, that this is an incorrect > conclusion to draw. > > This would > > > > > still be an abnormal termination. But that doesn't > > necessarily take away > > > > > the optimization potential at the RMD (assuming all the > > messages upto > > > > > the LastMessage are received). > > > > > > > > I'm still not following you. What resources do > you think the > > RMD can > > > > shed based on LastMessage? I think the situation you're > > focusing on is > > > > one where all messages have been received by the > RMD. So, in > > that case > > > > all you need to keep track of is the highest message number > > received, and > > > > note that you don't really need to keep around > anything else - > > just the > > > > number itself. Whether or not this one is tagged > as the last > > message > > > > doesn't change that nor can the RMD do anything with that > > information > > > > aside from not allow any message with a higher # > (as mentioned > > above). > > > > But in terms of freeing resources whether or not > the RMD can > > free any > > > > resources is not influenced by this additional flag. > > > > > > > > > > > > The LastMessage marker has no impact on the > > termination of the > > > > > > > > sequence. Even > > > > > > > > if the RMD gets a LastMessage marker but not a > > Terminate, and it > > > > > > chooses > > > > > > > > to timeout > > > > > > > > the sequence, its still an abnormal termination. > > > > > > > > > > > > > > Quite true, because the RMS may not have > received all > > the Acks. > > > > But the > > > > > > > RMD may not bother to do anything about it > and leave it > > to the > > > > RMS to > > > > > > > take correctly action, if necessary. Why is it that > > only the RMS can > > > > > > > initiate corrective action (as you state below)? > > Wouldn't that > > > > depend on > > > > > > > the application/context. > > > > > > > > > > > > Any corrective action an RMD can take (which > I think is > > limited to > > > > > > closing or terminating a sequence) is not based on the > > LastMessage > > > > > > tag/marker. > > > > > > > > > > > > > > > > Why is that? > > > > > If the RMD receives all the messages in a > Sequence (and it > > knows that > > > > > from the LM marker), and there is a abnormal termination > > (cause the RMD > > > > > did not receive the TerminateSequence message), RMD may > > decide not to > > > > > worry about any corrective action. OTOH, the > RMD may decide > > to take a > > > > > corrective action if it knows that it hasn't > received all > > the messages > > > > > in the Sequence (and LM is the key in figuring > that out). > > > > > > > > Ok, so if a sequence has 3 messages there are two > situations, > > one where > > > > the LM was sent and one where it was not. And > then, in both > > cases, there > > > > is an abnormal termination. I claim that the > behavior of the > > RMS and RMD > > > > in both cases would be the same. The RMS, whether > or not the > > LM was used, > > > > with either be ok with the ACKs it had received or > not. On the > > RMD side > > > > I think you're trying to say that the RMD can > think of it as > > being "less > > > > serious" if the highest message was tagged with > the LM marker > > and therefore > > > > could just drop the sequence, right? > > > > I don't think you can make this assumption though. > Since the > > RMD has not > > > > received a terminate the only safe thing it can > assume is that > > the RMS > > > > has not received all of the ACKs - so while I > think what you're > > saying is > > > > that the RMD can drop the seq since it got the LM > marker, it > > can't since > > > > it can't make any assuption about what the RMS knows. And, > > therefore would > > > > not be able to drop any more knowledge of the > sequence since it > > must still > > > > respond to Close/Terminate messages. > > > > > > > > > > > > The only non-abnormal > > > > > > > > termination > > > > > > > > is when the RMD receives a Terminate > from the RMS. > > And,either > > > > way, > > > > > > the > > > > > > > > RMD needs > > > > > > > > to keep the sequence state around until it gets a > > Terminate or it > > > > > > > > decides to terminate it > > > > > > > > on its own. Receipt of a LastMessage > does not change > > this. > > > > > > > > > > > > > > But the RMD can relinquish the message > store/queue at this > > > > point. All it > > > > > > > has to remember is the Seq Number for the > LastMessage. > > > > > > > > > > > > What the RMD needs to remember isn't dependent on > > LastMessage. > > > > Let's say > > > > > > the RMS sends messages 1, 2 and 3 but 3 is > lost. The RMD > > will > > > > remember > > > > > > that > > > > > > 1 and 2 arrived. Upon receipt of an AckReq > > (w/LastMessage=3) what will > > > > > > it do > > > > > > with this knowledge? Nothing. It will still > send back an > > Ack with > > > > message > > > > > > numbers 1 and 2. Knowing that the RMS sent > any message # > > higher > > > > than 2 > > > > > > doesn't > > > > > > change the state information of what the RMD > needs to keep > > around. > > > > How > > > > > > much > > > > > > of each message the RMD needs to remember > (just state vs > > entire > > > > message > > > > > > content) > > > > > > is more tied to the InOrder flag - so it > needs to remember > > the entire > > > > > > message > > > > > > until the time it can actually deliver the > message to the > > RMD's app. > > > > > > Again, > > > > > > LastMessage has no impact on this. > > > > > > > > > > > > > > > > I didn't quite state the situation correctly: > > > > > The RMD can relinquish the message store/queue > only when it > > knows it has > > > > > received all the messages. This requires that > there are no > > 'holes' *and* > > > > > the knowledge of the seqno of the last message. > > > > > I.e., if a message within the sequence was not received > > (received > > > > > messages: 1, 2, 3, 5, 6) then knowing that 6 > was the last > > message does > > > > > not provide any ability for the RMD to optimize. > > > > > > > > See above - previous comment. > > > > > > > > > > > > Ultimately, > > > > > > > > its up to the > > > > > > > > RMS to decide whether or not a sequence > needs to have > > some > > > > kind of > > > > > > error > > > > > > > > recovery > > > > > > > > done, not the RMD, and this would be based on the > > Acks it > > > > receives and > > > > > > > > not the delivery > > > > > > > > of a Terminate (or LastMessage marker) > to the RMD. > > > > > > > > > > > > > > > > As to freeing up some resources, the > LastMessage > > doesn't > > > > change this > > > > > > > > either. The data > > > > > > > > an RMD retains isn't dependent on the > LastMessage. > > You seem to > > > > > > indicate > > > > > > > > that the RMD > > > > > > > > can free up some stuff base on this marker - this > > isn't true. > > > > > > > > > > > > > > Any reason not to relinquish the message > store, close > > > > > > > connections/sessions, commit transactions etc? > > > > > > > > > > > > Perhaps but its not based on LastMessage but > rather other > > information, > > > > > > like which messages have arrived and been > delivered (as > > mentioned > > > > above). > > > > > > > > > > > > > > > > So if the RMD has received all the messages > (including the > > last message > > > > > in the Sequence) and if the LM marker is not > present the RMD > > cannot > > > > > relinquish the message store/connection/session > etc unless > > either it > > > > > timesout or expires (and not all Sequences have > expiration > > time) or the > > > > > TerminateSequence message (which is unreliable) > is received. > > > > > > > > See above. > > > > > > > > > > > I'm assuming that the motivation for getting rid of > > LastMessage > > > > marker > > > > > > > is so that the RMS is not forced to send > an additional > > empty message > > > > > > > when it is done. For some applications, it > is not clear > > which > > > > the last > > > > > > > message with payload is, until it has > finished sending > > that message. > > > > > > > > > > > > I don't think it matters. If after message > #3 is sent, > > the RMS decides > > > > > > that that really was the last message then it > can send a > > Terminate > > > > once it > > > > > > receives the Ack for all messages (1-3). > Having to send > > an additional > > > > > > message > > > > > > to say 'tag message 3 as the last one' > doesn't change any > > message > > > > > > processing > > > > > > logic on either side. In either case the RMD > will still > > need to > > > > > > maintain the > > > > > > exact same state information. I'm claiming that any > > information > > > > that you > > > > > > think the RMD can relinquish upon receipt of the > > LastMessage could have > > > > > > been dropped before the receipt of the > LastMessage marker. > > > > > > > > > > > > > > > > The RMD cannot drop the message store without > the LM marker > > (assuming > > > > > that there are no 'holes') unless it receives a > > TerminateSequence > > > > > message or timesout/expires. > > > > > > > > See above - LM marker doesn't change anything. > > > > > > > > > > > What if we were to make the LastMessage marker > > optional? Allow > > > > RMS to > > > > > > > include it and the RMD can optimize things > based on the > > marker if it > > > > > > > wishes to do so, but not require the RMS > to always send it. > > > > > > > > > > > > I still don't see what optimizations you > think the RMD can > > do with it? > > > > > > I don't think there are any - which is why I > think its a > > useless > > > > feature. >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]