[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [ws-rx] NEW ISSUE: Remove LastMessage
Lei, Yes that would indeed work, but there are some scalability/response-time issue that have to be compromised for this. Another possibility is to not use PTOS (where this is a concern) and use reasonable expiration times. In any case, I do agree that this is not that big a concern (since the advantages are scoped to only specific situations, which can be alleviated by other means). Thx! -Anish -- Lei Jin wrote: > Anish: > > I still feel this concern is implementation specific. First of all, > at least in our implementation, maintaining the store/queue is not > costly if we can get rid of all the messages belonging to that > particular sequence. While I understand this may not be true in other > implementations, I can't imagine this being a significant cost. > Secondly, if maintaining a store will cause a significant resource leak, > you can always have local implementation strategy to get rid of the > store after a period of no message received for that sequence(and there > is no gap in the received messages). This presumably will take care of > the situation where a TerminateSequence is missing. If you do receive a > message later, you can always recreate the store. > > Lei > > >>-----Original Message----- >>From: Anish Karmarkar [mailto:Anish.Karmarkar@oracle.com] >>Sent: Wednesday, November 30, 2005 6:59 PM >>To: ws-rx@lists.oasis-open.org >>Subject: Re: [ws-rx] NEW ISSUE: Remove LastMessage >> >> >>Lei Jin wrote: >> >>>I think Anish is talking about releasing the actual store, not the >>>individual messages inside of it. This concern seems too >> >>implementation >> >>>specific to me. >>> >> >>Right. >> >>I'm not sure that this is too implementation specific. It is >>certainly >>possible to build implementations that won't care about such an >>optimization. But this really depends on the operating environment. >> >>It is possible that we (or I) are really looking at premature >>optimization (which per Knuth is the root of all evil) here. >>But I'm not convinced (yet) that this is indeed so. >> >>Given that TerminateSequence is unreliable (for which Bob/Tom has >>already raised an issue -- which if fixed makes my concern >>marginal) and >>that the expiration time can be PTOS, this may indeed be a >>problem. This >>is sort of similar to the outcome of the InactivityTimeout >>discussion, >>where we decided that the inactivity timeout did not make sense but a >>timeout for the 1st message to arrive after CS/CSR exchange did. This >>was because the CSR message may be lost resulting in an >>orphaned sequence. >> >>Cheers! >> >>-Anish >>-- >> >> >>>Lei >>> >>> -----Original Message----- >>> *From:* Doug Davis [mailto:dug@us.ibm.com] >>> *Sent:* Wednesday, November 30, 2005 4:09 PM >>> *To:* ws-rx@lists.oasis-open.org >>> *Subject:* Re: [ws-rx] NEW ISSUE: Remove LastMessage >>> >>> >>> Anish, >>> more inline... >>> thanks >>> -Doug >>> >>> >>> Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote on 11/30/2005 >>> 06:16:53 PM: >>> >>> > Ok, perhaps I'm not being clear about my concern. >> >>Apologies if I'm >> >>> > completely missing your point. >>> > >>> > 1) I'm *not* worried about the security issue. I >> >>agree with you that >> >>> > using the LM marker to prevent hijacking of the >> >>sequence is an >> >>> > inadequate solution. People should not rely on that. >>> > >>> > 2) I'm *not* saying that if the RMD receives a >> >>message with a LM >> >>> marker >>> > and it has received all the messages in the sequence >> >>that it can >> >>> release >>> > *all* its resources. It still has to hang around to >> >>ack messages >> >>> and can >>> > only terminate the sequence when it receives the >> >>TerminateSequence >> >>> > message. Without that (TerminateSequence) the RMD >> >>cannot make any >> >>> > determination about which acks have been received by the RMS. >>> > >>> > 3) What I'm saying is the following - >>> > When: >>> > a) an LM marker is present in the last message, and >>> > b) the RMD receives all the messages upto and including the >>> message with >>> > the LM marker. I.e. all the messages in the range [1, >>> Seqno(LM-message)] >>> > then: >>> > at that point the RMD knows that it has received all >> >>the messages >> >>> in the >>> > sequence. In certain implementations it can make the >>> determination that >>> > it no longer needs the message store (it still needs >> >>to be around >> >>> to ack >>> > messages) -- assuming that the messages have been >> >>successfully >> >>> delivered >>> > to the AD. i.e., it can give up its message store resource at >>> this point >>> > but *not* terminate the Sequence. >>> >>> Why would it need to keep them around w/o the LM marker? Once a >>> message >>> has been delivered to the AD why would the RMD need to keep it >>> around at >>> all - regardless of the LM marker? >>> >>> > Without the LM marker, it will never know if it has >> >>received all the >> >>> > messages in the Sequence until it receives the >> >>TerminateSequence >> >>> message >>> > (which is unreliable). >>> >>> Under what situations would the RMD not deliver a >> >>message to the AD >> >>> when it >>> has all of the messages from 1 to X and then NOT clear >> >>up its store >> >>> for those >>> messages? Once it has 1 it can deliver it and erase >> >>it, once it did >> >>> that >>> it can do it for 2, then 3.... marking X as the LM >> >>doesn't change that. >> >>> > To take the previous example, there are 3 messages >> >>in the sequence - >> >>> > >>> > scenario A: LM marker present in message # 3, RMD >> >>has received >> >>> all the >>> > messages (1, 2, 3) but not the TerminateSequence. >> >>RMD knows that >> >>> there >>> > will never ever be message number 4 (and beyond) and >> >>therefore >> >>> does not >>> > need the message store (but still needs to hang >> >>around to ack). >> >>> Doesn't need it for 1,2 and 3 at all anyway since >> >>there's nothing >> >>> stopping >>> it from deliverying all 3 to the AD and erasing it from >> >>its store. >> >>> > scenario B: LM marker is not present in message # 3, RMD has >>> received >>> > all the messages (1, 2, 3) but not the TerminateSequence. RMD >>> does not >>> > know that there won't be a message number 4 (and beyond) and >>> therefore >>> > may indeed need the message store (say if 5 were to >> >>arrive before >> >>> 4 and >>> > the DA was inOrder, it will have to store 5 till it >> >>receives 4 >> >>> and then >>> > delivery both). >>> >>> But it can still deliver and erase 1, 2 and 3 - same as >> >>above. LM >> >>> doesn't >>> change that. >>> >>> > So I think that the LM marker does allow the RMD to >> >>free up certain >> >>> > (though not all) resources when it has received all >> >>the messages >> >>> in the >>> > sequence and before the TerminateSequence message is received >>> (this of >>> > course would depend on the implementation, YMMV). >>> > >>> > Hope that makes sense. >>> > >>> > Thx. >>> > >>> > -Anish >>> > -- >>> > >>> > Doug Davis wrote: >>> > > >>> > > Anish, >>> > > more inline - I think we're getting closer :-) >>> > > -Doug >>> > > >>> > > >>> > > Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote >> >>on 11/29/2005 >> >>> > > 10:43:05 PM: >>> > > >>> > > > Hey Doug, >>> > > > >>> > > > More comments inlined below. >>> > > > >>> > > > I realize that I'm picking on an optimization >> >>feature which is >> >>> > > > applicable only a certain scenario. But I want >> >>to explore >> >>> this to ensure >>> > > > that we are indeed doing the right thing. One >> >>of the reasons I'm >> >>> > > > concerned about this optimization feature is because a >>> Sequence may not >>> > > > have an expiration time (PTOS), in which case >> >>the loss of >> >>> the unreliable >>> > > > SequenceTermination message may result in the >> >>RMD holding on >> >>> to the >>> > > > message queue much longer than needed. More below. >>> > > > Thx! >>> > > > >>> > > > -Anish >>> > > > -- >>> > > > >>> > > > Doug Davis wrote: >>> > > > > >>> > > > > Anish, >>> > > > > comment inline. >>> > > > > thanks >>> > > > > -Doug >>> > > > > >>> > > > > Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote on >>> 11/28/2005 >>> > > > > 09:48:10 PM: >>> > > > > >>> > > > > > I thought this issue arose because of the >> >>addition of >> >>> the Close >>> > > message >>> > > > > > and I'm not sure why the Close message >> >>changes anything >> >>> wrt >>> > > LastMessage >>> > > > > > marker. >>> > > > > >>> > > > > I actually agree. I don't think Close >> >>changes the need >> >>> for (or >>> > > lack of >>> > > > > a need >>> > > > > for) LastMessage but if it help convince >> >>others then ok ;-) >> >>> > > > > >>> > > > > > > I think your argument actually adds to >> >>the reasons >> >>> why it >>> > > should be >>> > > > > > > removed :-) >>> > > > > > > You're placing some additional semantics on the >>> LastMessage >>> > > marker >>> > > > > that >>> > > > > > > are not in the >>> > > > > > > spec and removing this marker would remove the >>> possibility of >>> > > others >>> > > > > > > reading too much >>> > > > > > > into it too. >>> > > > > > You are right that the spec does not explicitly say >>> that but it >>> > > seems >>> > > > > > like a corollary to what the spec says. >>> > > > > >>> > > > > Not to me - and this ambiguity is part of the >> >>reason to >> >>> remove it. >>> > > > > >>> > > > >>> > > > Not sure what additional semantics I'm placing on >>> LastMessage marker. >>> > > > If I get a message with LastMessage marker, I >> >>know that this >> >>> is the last >>> > > > message in the sequence => RMD cannot get a message with >>> SeqNo larger >>> > > > than the SeqNo of the message containing >> >>LastMessage marker. >> >>> > > >>> > > Yes - as I noted in the original issue text this would just >>> prevent the >>> > > RMD from receiving message with a higher #, but >> >>that's about >> >>> it. So, >>> > > if security is the reason for this feature, then >> >>its a poor way >> >>> of adding >>> > > security and the other means (SC/Trust) are far >> >>more appropriate. >> >>> > > >>> > > > This allows >>> > > > the RMD to shed some resources, if (and only if) it has >>> received all the >>> > > > messages between SeqNo 1 and the SeqNo of the >> >>LastMessage. >> >>> > > > >>> > > > Perhaps you are referring to my email where I say: >>> > > > "... In this case, the loss of the >> >>TerminateSequence message >> >>> does not >>> > > > result in an abnormal termination of the >> >>sequence on the RMD >> >>> side on >>> > > > time out and no corrective action is necessary >> >>on the RMD side." >> >>> > > > >>> > > > You are right, that this is an incorrect >> >>conclusion to draw. >> >>> This would >>> > > > still be an abnormal termination. But that doesn't >>> necessarily take away >>> > > > the optimization potential at the RMD (assuming all the >>> messages upto >>> > > > the LastMessage are received). >>> > > >>> > > I'm still not following you. What resources do >> >>you think the >> >>> RMD can >>> > > shed based on LastMessage? I think the situation you're >>> focusing on is >>> > > one where all messages have been received by the >> >>RMD. So, in >> >>> that case >>> > > all you need to keep track of is the highest message number >>> received, and >>> > > note that you don't really need to keep around >> >>anything else - >> >>> just the >>> > > number itself. Whether or not this one is tagged >> >>as the last >> >>> message >>> > > doesn't change that nor can the RMD do anything with that >>> information >>> > > aside from not allow any message with a higher # >> >>(as mentioned >> >>> above). >>> > > But in terms of freeing resources whether or not >> >>the RMD can >> >>> free any >>> > > resources is not influenced by this additional flag. >>> > > >>> > > > > > > The LastMessage marker has no impact on the >>> termination of the >>> > > > > > > sequence. Even >>> > > > > > > if the RMD gets a LastMessage marker but not a >>> Terminate, and it >>> > > > > chooses >>> > > > > > > to timeout >>> > > > > > > the sequence, its still an abnormal termination. >>> > > > > > >>> > > > > > Quite true, because the RMS may not have >> >>received all >> >>> the Acks. >>> > > But the >>> > > > > > RMD may not bother to do anything about it >> >>and leave it >> >>> to the >>> > > RMS to >>> > > > > > take correctly action, if necessary. Why is it that >>> only the RMS can >>> > > > > > initiate corrective action (as you state below)? >>> Wouldn't that >>> > > depend on >>> > > > > > the application/context. >>> > > > > >>> > > > > Any corrective action an RMD can take (which >> >>I think is >> >>> limited to >>> > > > > closing or terminating a sequence) is not based on the >>> LastMessage >>> > > > > tag/marker. >>> > > > > >>> > > > >>> > > > Why is that? >>> > > > If the RMD receives all the messages in a >> >>Sequence (and it >> >>> knows that >>> > > > from the LM marker), and there is a abnormal termination >>> (cause the RMD >>> > > > did not receive the TerminateSequence message), RMD may >>> decide not to >>> > > > worry about any corrective action. OTOH, the >> >>RMD may decide >> >>> to take a >>> > > > corrective action if it knows that it hasn't >> >>received all >> >>> the messages >>> > > > in the Sequence (and LM is the key in figuring >> >>that out). >> >>> > > >>> > > Ok, so if a sequence has 3 messages there are two >> >>situations, >> >>> one where >>> > > the LM was sent and one where it was not. And >> >>then, in both >> >>> cases, there >>> > > is an abnormal termination. I claim that the >> >>behavior of the >> >>> RMS and RMD >>> > > in both cases would be the same. The RMS, whether >> >>or not the >> >>> LM was used, >>> > > with either be ok with the ACKs it had received or >> >>not. On the >> >>> RMD side >>> > > I think you're trying to say that the RMD can >> >>think of it as >> >>> being "less >>> > > serious" if the highest message was tagged with >> >>the LM marker >> >>> and therefore >>> > > could just drop the sequence, right? >>> > > I don't think you can make this assumption though. >> >> Since the >> >>> RMD has not >>> > > received a terminate the only safe thing it can >> >>assume is that >> >>> the RMS >>> > > has not received all of the ACKs - so while I >> >>think what you're >> >>> saying is >>> > > that the RMD can drop the seq since it got the LM >> >>marker, it >> >>> can't since >>> > > it can't make any assuption about what the RMS knows. And, >>> therefore would >>> > > not be able to drop any more knowledge of the >> >>sequence since it >> >>> must still >>> > > respond to Close/Terminate messages. >>> > > >>> > > > > > > The only non-abnormal >>> > > > > > > termination >>> > > > > > > is when the RMD receives a Terminate >> >>from the RMS. >> >>> And,either >>> > > way, >>> > > > > the >>> > > > > > > RMD needs >>> > > > > > > to keep the sequence state around until it gets a >>> Terminate or it >>> > > > > > > decides to terminate it >>> > > > > > > on its own. Receipt of a LastMessage >> >>does not change >> >>> this. >>> > > > > > >>> > > > > > But the RMD can relinquish the message >> >>store/queue at this >> >>> > > point. All it >>> > > > > > has to remember is the Seq Number for the >> >>LastMessage. >> >>> > > > > >>> > > > > What the RMD needs to remember isn't dependent on >>> LastMessage. >>> > > Let's say >>> > > > > the RMS sends messages 1, 2 and 3 but 3 is >> >>lost. The RMD >> >>> will >>> > > remember >>> > > > > that >>> > > > > 1 and 2 arrived. Upon receipt of an AckReq >>> (w/LastMessage=3) what will >>> > > > > it do >>> > > > > with this knowledge? Nothing. It will still >> >>send back an >> >>> Ack with >>> > > message >>> > > > > numbers 1 and 2. Knowing that the RMS sent >> >>any message # >> >>> higher >>> > > than 2 >>> > > > > doesn't >>> > > > > change the state information of what the RMD >> >>needs to keep >> >>> around. >>> > > How >>> > > > > much >>> > > > > of each message the RMD needs to remember >> >>(just state vs >> >>> entire >>> > > message >>> > > > > content) >>> > > > > is more tied to the InOrder flag - so it >> >>needs to remember >> >>> the entire >>> > > > > message >>> > > > > until the time it can actually deliver the >> >>message to the >> >>> RMD's app. >>> > > > > Again, >>> > > > > LastMessage has no impact on this. >>> > > > > >>> > > > >>> > > > I didn't quite state the situation correctly: >>> > > > The RMD can relinquish the message store/queue >> >>only when it >> >>> knows it has >>> > > > received all the messages. This requires that >> >>there are no >> >>> 'holes' *and* >>> > > > the knowledge of the seqno of the last message. >>> > > > I.e., if a message within the sequence was not received >>> (received >>> > > > messages: 1, 2, 3, 5, 6) then knowing that 6 >> >>was the last >> >>> message does >>> > > > not provide any ability for the RMD to optimize. >>> > > >>> > > See above - previous comment. >>> > > >>> > > > > > > Ultimately, >>> > > > > > > its up to the >>> > > > > > > RMS to decide whether or not a sequence >> >>needs to have >> >>> some >>> > > kind of >>> > > > > error >>> > > > > > > recovery >>> > > > > > > done, not the RMD, and this would be based on the >>> Acks it >>> > > receives and >>> > > > > > > not the delivery >>> > > > > > > of a Terminate (or LastMessage marker) >> >>to the RMD. >> >>> > > > > > > >>> > > > > > > As to freeing up some resources, the >> >>LastMessage >> >>> doesn't >>> > > change this >>> > > > > > > either. The data >>> > > > > > > an RMD retains isn't dependent on the >> >>LastMessage. >> >>> You seem to >>> > > > > indicate >>> > > > > > > that the RMD >>> > > > > > > can free up some stuff base on this marker - this >>> isn't true. >>> > > > > > >>> > > > > > Any reason not to relinquish the message >> >>store, close >> >>> > > > > > connections/sessions, commit transactions etc? >>> > > > > >>> > > > > Perhaps but its not based on LastMessage but >> >>rather other >> >>> information, >>> > > > > like which messages have arrived and been >> >>delivered (as >> >>> mentioned >>> > > above). >>> > > > > >>> > > > >>> > > > So if the RMD has received all the messages >> >>(including the >> >>> last message >>> > > > in the Sequence) and if the LM marker is not >> >>present the RMD >> >>> cannot >>> > > > relinquish the message store/connection/session >> >>etc unless >> >>> either it >>> > > > timesout or expires (and not all Sequences have >> >>expiration >> >>> time) or the >>> > > > TerminateSequence message (which is unreliable) >> >>is received. >> >>> > > >>> > > See above. >>> > > >>> > > > > > I'm assuming that the motivation for getting rid of >>> LastMessage >>> > > marker >>> > > > > > is so that the RMS is not forced to send >> >>an additional >> >>> empty message >>> > > > > > when it is done. For some applications, it >> >>is not clear >> >>> which >>> > > the last >>> > > > > > message with payload is, until it has >> >>finished sending >> >>> that message. >>> > > > > >>> > > > > I don't think it matters. If after message >> >>#3 is sent, >> >>> the RMS decides >>> > > > > that that really was the last message then it >> >>can send a >> >>> Terminate >>> > > once it >>> > > > > receives the Ack for all messages (1-3). >> >>Having to send >> >>> an additional >>> > > > > message >>> > > > > to say 'tag message 3 as the last one' >> >>doesn't change any >> >>> message >>> > > > > processing >>> > > > > logic on either side. In either case the RMD >> >>will still >> >>> need to >>> > > > > maintain the >>> > > > > exact same state information. I'm claiming that any >>> information >>> > > that you >>> > > > > think the RMD can relinquish upon receipt of the >>> LastMessage could have >>> > > > > been dropped before the receipt of the >> >>LastMessage marker. >> >>> > > > > >>> > > > >>> > > > The RMD cannot drop the message store without >> >>the LM marker >> >>> (assuming >>> > > > that there are no 'holes') unless it receives a >>> TerminateSequence >>> > > > message or timesout/expires. >>> > > >>> > > See above - LM marker doesn't change anything. >>> > > >>> > > > > > What if we were to make the LastMessage marker >>> optional? Allow >>> > > RMS to >>> > > > > > include it and the RMD can optimize things >> >>based on the >> >>> marker if it >>> > > > > > wishes to do so, but not require the RMS >> >>to always send it. >> >>> > > > > >>> > > > > I still don't see what optimizations you >> >>think the RMD can >> >>> do with it? >>> > > > > I don't think there are any - which is why I >> >>think its a >> >>> useless >>> > > feature. >>
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]